The Pathmind Helper is an AnyLogic palette item that enables you to apply reinforcement learning to your simulations. It serves two purposes:

  • Exposes simulation data and converts it into a format that a reinforcement learning algorithm can consume for training.  
  • Enables you to query a trained policy in AnyLogic.

Installation

Step 1: Download the Pathmind Helper (214 MB). 

Step 2: Add the Pathmind Helper JAR file as an AnyLogic palette item.

Step 3:  Drag and drop the Pathmind Helper palette item into your top-level agent (typically "Main", which is the AnyLogic default)*. 

*There can only be a single instance of the Pathmind Helper per AnyLogic model. Adding more than one helper will result in an error. 

Pathmind Helper Properties

Click on the Pathmind Helper to view its properties. 

Top-Level Properties

[IMPORTANT] Single Agent vs Population of Agents 

These radio buttons are default AnyLogic options (i.e. they can't be removed). Although these options do nothing in the context of Pathmind, you must always select "Single Agent" for Pathmind to work.  

Debug Mode

A toggle to print simulation data passed to Pathmind Helper. This is helpful for auditing purposes.

Mode
Use Random Actions tells Pathmind to select actions at random. This option serves two purposes:

  1. Confirm that your action space is configured correctly. If your agent does nothing, there is likely a logic error in your simulation. 
  2. Construct a baseline to measure the performance of a trained policy. Better than random is generally a good starting point. 

Use Policy  will execute your simulation using the trained policy obtained from Pathmind instead of random actions. When Use Policy is selected, the Pathmind Helper will query the policy to predict the next best action.

Reinforcement Learning Values

Number of Agents is defined as the total number of controlled agents in your simulation. For the majority of use cases, a single agent is adequate.

Observations should contain anything that is relevant for an agent to make an informed decision.

Reward Variables determine what feedback an agent will receive after each action is taken.

Actions are a list of all possible actions that an agent can perform. 

Event Trigger 

The event trigger tells an agent when it should execute an action. The event trigger must return true (execute action) or false (do not execute action).

Miscellaneous  

Please ignore everything below Event Trigger. 

  • Dimensions and Movement
  • Initial Location
  • Statistics
  • Advanced 
  • Description

These are default AnyLogic options that cannot be removed.

Did this answer your question?