Sequential Agents

Built on top of LangGraph, Sequential Agents in Tailwinds represent a groundbreaking approach to creating complex, intelligent AI workflows. This feature allows you to chain multiple AI agents together, creating sophisticated processes that can handle intricate tasks with greater efficiency and flexibility than ever before.

KNOWN ISSUES

• Sequential Agents only work with Chat based LLM models (not instruct)

• Sequential "LLM Nodes" works with all Chat models

• Sequential Agents only work with OpenAI at this time.

Key Features of Sequential Agents

1. Agent Chaining: Connect multiple AI agents to create a seamless workflow.

2. Loopback Mechanisms: Enable agents to revisit and refine their processes.

3. Human-in-the-Loop Integration: Incorporate human oversight and decision-making at crucial points.

4. Conditional Branching: Create dynamic workflows that adapt based on specific conditions or outcomes.

Benefits of Using Sequential Agents

• Enhanced Problem-Solving: Tackle complex issues by breaking them down into manageable steps handled by specialized agents.

• Improved Accuracy: Utilize multiple agents to cross-check and refine results.

• Greater Flexibility: Easily modify workflows to adapt to changing requirements or to optimize performance.

• Scalability: Handle increasingly complex tasks by adding or modifying agents in the workflow.

Best Practices

• Break down the process into distinct steps or decision points.

• Start with simple workflows and gradually increase complexity.

• Use clear, specific prompts for each agent to ensure focused task execution.

• Implement error handling and fallback options in your workflows.

• Regularly review and optimize your workflows based on performance data.

---

Introducing State, Loop and Conditional Nodes

Sequential Agents offer new capabilities for creating conversational systems that can adapt to user input, make decisions based on context, and perform iterative tasks.

These new capabilities are made possible by the introduction of four core nodes; the State Node, the Loop Node, and two Conditional Nodes.

• State Node: We define State as a shared data structure that represents the current snapshot of our application or workflow. The State Node allows us to add a custom State to our workflow from the start of the conversation. This custom State is accessible and modifiable by other nodes in the workflow, enabling dynamic behavior and data sharing.

• Loop Node: This node introduces controlled cycles within the Sequential Agent workflow, enabling iterative processes where a sequence of nodes can be repeated based on specific conditions. This allows agents to refine outputs, gather additional information from the user, or perform tasks multiple times.

• Conditional Nodes: The Conditional and Condition Agent Node provide the necessary control to create complex conversational flows with branching paths. The Conditional Node evaluates conditions directly, while the Condition Agent Node uses an agent's reasoning to determine the branching logic. This allows us to dynamically guide the flow's behavior based on user input, the custom State, or results of actions taken by other nodes.

Choosing the right system

The choice of the correct system always depends on the specific needs of your application workflow. Consider factors such as task complexity, the need for parallel node execution, and the desired level of control over data flow.

• For simplicity: If your workflow is relatively straightforward, where tasks can be completed one after the other and therefore does not require parallel node execution or Human-in-the-Loop (HITL), the Multi-Agent approach offers ease of use and quick setup.

• For flexibility: If your workflow needs parallel execution, dynamic conversations, custom State management, and the ability to incorporate HITL, the Sequential Agent approach provides the necessary flexibility and granular control.

Here's a table comparing key differences between Multi-Agent and Sequential Agent implementations:

	Multi-Agent	Sequential Agent
Structure	Hierarchical; Supervisor delegates to specialized Workers.	Linear, cyclic and/or branching; nodes connect in a sequence, with conditional logic for branching.
Workflow	Flexible; designed for breaking down a complex task into a sequence of sub-tasks, completed one after another.	Highly flexible; supports parallel node execution, complex dialogue flows, branching logic, and loops within a single conversation turn.
Parallel Node Execution	No; Supervisor handles one task at a time.	Yes; can trigger multiple actions (Agent Nodes, LLM Nodes, Tool Nodes, etc) in parallel within a single run.
State Management	Implicit; State is in place, but is not explicitly managed by the developer.	Explicit; State is in place, and developers can define and manage an initial or custom State using the State Node and the "Update State" field in various nodes.
Tool Usage	Workers can access and use tools as needed.	Tools are accessed and executed through Agent Nodes and Tool Nodes.
Human-in-the-Loop (HITL)	HITL is not supported.	Supported through the Agent Node and Tool Node's "Require Approval" feature, allowing human review and approval or rejection of tool execution.
Complexity	Higher level of abstraction; simplifies workflow design.	Lower level of abstraction; more complex workflow design, requiring careful planning of node interactions, state management, and conditional logic.
Ideal Use Cases	Automating linear processes (e.g., data extraction, lead generation). Situations where sub-tasks need to be completed one after the other.	Building conversational systems with dynamic flows. Complex workflows requiring parallel node execution or branching logic. Situations where decision-making is needed at multiple points in the conversation.

Important note: Even though Multi-Agent systems are technically a higher-level layer built upon the Sequential Agent architecture, they offer a distinct user experience and approach to workflow design. This above section compares them as separate systems to help you choose the best option for your needs.

---

1. Start Node

As the name suggests, the Start Node serves as the entry point for all workflows in the Sequential Agent architecture. It's where the initial user query is first registered, setting the Stage for the flow and is responsible for initializing the conversation.

Understanding the Start Node

The Start Node ensures that our conversational workflows have the necessary setup and context to function correctly. It's responsible for setting up key functionalities that will be used throughout the rest of the workflow:

• Defining the default LLM: The Start Node requires us to specify a Chat Model (LLM) compatible with function calling, enabling agents in the workflow to interact with tools and external systems. It will be the default LLM used under the hood in the workflow.

• Initializing Memory: We can optionally include an Agent Memory Node to store and retrieve conversation history, enabling more context-aware responses.

• Setting a custom State: Setting a custom State: By default, the State contains a state.messages array, acting as the transcript or history of the conversation between the user and the agents. The Start Node allows us to set a custom State to the workflow by adding a State Node. This enables storing additional information relevant to the specific workflow.

• Enabling moderation: Optionally, we can enable Input Moderation to analyze the user's input and prevent potentially harmful content from being sent to the LLM.

Inputs

	Required	Description
Chat Model	Yes	The default LLM that will power the conversation. Only compatible with models that are capable of function calling.
Agent Memory Node	No	Connect an Agent Memory Node to enable persistence and context preservation.
State Node	No	Connect a State Node to set a custom State, a shared context that can be accessed and modified by other nodes in the workflow.
Input Moderation	No	Connect a Moderation Node to filter content by detecting text that could generate harmful output, preventing it from being sent to the language model.

Outputs

The Start Node can connect to the following nodes as outputs:

• Agent Node: Routes the conversation flow to an Agent Node, which can then execute actions or access tools based on the conversation's context.

• LLM Node: Routes the conversation flow to an LLM Node for processing and response generation.

• Condition Agent Node: Connects to a Condition Agent Node to introduce conditional logic based on the agent's evaluation of the conversation.

• Condition Node: Connects to a Condition Node to implement branching logic based on predefined conditions.

Best Practices

Pro Tips

Choose the right Chat Model

Ensure your selected LLM supports function calling, a key feature for enabling agent-tool interactions. Additionally, choose an LLM that aligns with the complexity and requirements of your application. You can override the default LLM by setting it at the Agent/LLM/Conditional Agent node level when necessary.

Consider context and persistence

If your use case demands it, utilize Agent Memory Node to maintain context and personalize interactions.

Potential Pitfalls

Incomplete or inconsistent custom State initialization

• Problem: The initial State defined in the Start Node is missing crucial variables or contains incorrect or inconsistent data, leading to errors or unexpected behavior in subsequent nodes.

• Example: A workflow requires a "userName" variable to personalize responses, but the Start Node doesn't initialize this variable, causing errors in nodes that try to access it.

• Solution: Plan and define all necessary state variables in the Start Node, ensuring they have appropriate initial values and data types. Review the requirements of all downstream nodes to identify any essential state variables.

Incorrect Chat Model (LLM) selection

• Problem: The Chat Model selected in the Start Node is not suitable for the intended tasks or capabilities of the workflow, resulting in poor performance or inaccurate responses.

• Example: A workflow requires a Chat Model with strong summarization capabilities, but the Start Node selects a model optimized for code generation, leading to inadequate summaries.

• Solution: Choose a Chat Model that aligns with the specific requirements of your workflow. Consider the model's strengths, weaknesses, and the types of tasks it excels at. Refer to the documentation and experiment with different models to find the best fit.

Overlooking Agent Memory Node configuration

• Problem: The Agent Memory Node is not properly connected or configured, resulting in the loss of conversation history or state data between sessions.

• Example: You intend to use persistent memory to store user preferences, but the Agent Memory Node is not connected to the Start Node, causing preferences to be reset on each new conversation.

• Solution: Ensure that the Agent Memory Node is connected to the Start Node and configured with the appropriate database (SQLite).

Inadequate Input Moderation

• Problem: The "Input Moderation" option is not enabled or configured correctly, allowing potentially harmful or inappropriate user input to reach the LLM and generate undesirable responses.

• Example: A user submits offensive language, but the input moderation fails to detect it or is not set up at all, and the LLM incorporates the offensive content into its response.

• Solution: Add and configure an input moderation node in the Start Node to filter out potentially harmful or inappropriate language. Customize the moderation settings to align with your specific requirements and use cases.

2. Agent Memory Node

Unlike the State Node, which only holds custom State information for a single workflow execution, the Agent Memory Node provides a mechanism for persistent memory storage. This allows the Sequential Agent system to retain the conversation history state.messages and any user-defined state variables across multiple interactions within a workflow.

This long-tewrm memory is crucial for agents to learn from previous interactions, maintain context over extended conversations, and provide more relevant responses.

Where the data is recorded

A built-in SQLite database stores conversation history and custom state data by default, creating a "checkpoints" table to manage this persistent information.

Understanding the "checkpoints" table structure and data format

This table stores snapshots of the system's State at various points during a conversation, enabling the persistence and retrieval of conversation history and execution details. Each row represents a specific point or "checkpoint" in the workflow's execution.

Table structure

• thread_id: A unique identifier representing a specific conversation session, our session ID. It groups together all checkpoints related to a single workflow execution.

• checkpoint_id: A unique identifier for each execution step (node execution) within the workflow. It helps track the order of operations and identify the State at each step.

• parent_id: Indicates the checkpoint_id of the preceding execution step that led to the current checkpoint. This establishes a hierarchical relationship between checkpoints, allowing for the reconstruction of the workflow's execution flow.

• checkpoint: Contains a JSON string representing the current State of the workflow at that specific checkpoint. This includes the values of variables, the messages exchanged, and any other relevant data captured at that point in the execution.

• metadata: Provides additional context about the checkpoint, specifically related to node operations.

How it works

As a Sequential Agent workflow executes, the system records a checkpoint in this table for each significant step. This mechanism provides several benefits:

• Execution tracking: Checkpoints enable the system to understand the execution path and the order of operations within the workflow.

• State management: Checkpoints store the State of the workflow at each step, including variable values, conversation history, and any other relevant data. This allows the system to maintain contextual awareness and make informed decisions based on the current State.

• Workflow resumption: If the workflow is paused or interrupted (e.g., due to a system error or user request), the system can use the stored checkpoints to resume execution from the last recorded State. This ensures that the conversation or task continues from where it left off, preserving the user's progress and preventing data loss.

Inputs

The Agent Memory Node has no specific input connections.

Node Setup

	Required	Description
Database	Yes	The type of database used for storing conversation history. Currently, only SQLite is supported.

Additional Parameters

	Required	Description
Database File Path	No	The file path to the SQLite database file. If not provided, the system will use a default location.

Outputs

The Agent Memory Node interacts solely with the Start Node, making the conversation history available from the very beginning of the workflow.

Best Practices

Pro Tips

Strategic use

Employ Agent Memory only when necessary. For simple, stateless interactions, it might be overkill. Reserve it for scenarios where retaining information across turns or sessions is essential.

Potential Pitfalls

Unnecessary overhead

• The Problem: Using Agent Memory for every interaction, even when not needed, introduces unnecessary storage and processing overhead. This can slow down response times and increase resource consumption.

• Example: A simple weather chatbot that provides information based on a single user request doesn't need to store conversation history.

• Solution: Analyze the requirements of your system and only utilize Agent Memory when persistent data storage is essential for functionality or user experience.

---

3. State Node

The State Node, which can only be connected to the Start Node, provides a mechanism to set a user-defined or custom State into our workflow from the start of the conversation. This custom State is a JSON object that is shared and can be updated by nodes in the graph, passing from one node to another as the workflow progresses.

Understanding the State Node

By default, the State includes a state.messages array, which acts as our conversation history. This array stores all messages exchanged between the user and the agents (or any actors in the workflow). It is appended with each message sent and received, preserving the complete conversation history throughout the workflow execution.

So, beyond the default conversation history provided by state.messages, the State Node allows us to define custom key-value pairs, expanding the state object to hold any additional information relevant to our specific workflow.

When no Agent Memory Node is used, the current State operates in-memory and is not persisted for future use. It is also important to note that the state.messages array is immutable and cannot be modified or overwritten by users.

Inputs

The State Node has no specific input connections.

Outputs

The State Node can only connect to the Start Node, allowing the setup of a custom State from the beginning of the workflow and allowing other nodes to access and potentially modify this shared custom State.

Additional Parameters

Parameter	Required	Description
Custom State	Yes	A JSON object representing the initial custom State of the workflow. This object can contain any key-value pairs relevant to the application.

How to set a custom State

Specify the Key, Operation Type, and Default Value for the state object. The Operation Type can be either "Replace" or "Append".

• Replace

  1. Replace the existing value with the new value.

  2. If the new value is null, the existing value will be retained.

• Append

  1. Append the new value to the existing value.

  2. Default values can be empty or an array. Ex: ["a", "b"]

  3. Final value is an array.

Example using JS

{
    aggregate: {
        value: (x, y) => x.concat(y), // here we append the new message to the existing messages
        default: () => []
    }
}

Example using Table

To define a custom State using the table interface in the State Node, follow these steps:

1. Add rows: Click the "+ Add Item" button to add rows to the table. Each row represents a key-value pair in your custom State.

2. Specify keys: In the "Key" column, enter the name of each key you want to define in your state object. For example, you might have keys like "userName," "userLocation," or "cartItems."

3. Choose operations: In the "Operation" column, select the desired operation for each key. You have two options:

   • Replace: This will replace the existing value of the key with the new value provided by a node. If the new value is null, the existing value will be retained.

   • Append: This will append the new value to the existing value of the key. The final value will be an array.

4. Set default values: In the "Default Value" column, enter the initial value for each key. This value will be used if no other node provides a value for the key. The default value can be empty or an array. For example:

   • [] (an empty array)

   • ["a", "b"] (an array with initial values)

Example Table

Key	Operation	Default Value
userName	Replace	null

Explanation

• This table defines one key in the custom State: userName.

• The userName key will use the "Replace" operation, meaning its value will be updated whenever a node provides a new value.

• The userName key has a default value of null, indicating that it has no initial value.

Remember that this table-based approach is an alternative to defining the custom State using JavaScript code. Choose the method that best suits your needs and workflow complexity.

Best Practices

Pro-Tips

Plan your custom State structure

Before building your workflow, design the structure of your custom State. A well-organized custom State will make your workflow easier to understand, manage, and debug.

Use meaningful key names

Choose descriptive and consistent key names that clearly indicate the purpose of the data they hold. This will improve the readability of your code and make it easier for others (or you in the future) to understand how the custom State is being used.

Keep custom State minimal

Only store information in the custom State that is essential for the workflow's logic and decision-making.

Consider State persistence

If you need to preserve State across multiple conversation sessions (e.g., for user preferences, order history, etc.), use the Agent Memory Node to store the State in a persistent database.

Potential Pitfalls

Inconsistent State Updates

• Problem: Updating the state in multiple nodes without a clear strategy can lead to inconsistencies and unexpected behavior.

• Example

  1. Agent 1 updates orderStatus to "Payment Confirmed".

  2. Agent 2, in a different branch, updates orderStatus to "Order Complete" without checking the previous status.

• Solution: Use Conditional Nodes to control the flow of the custom State updates and ensure that custom State transitions happen in a logical and consistent manner.

---

4. Agent Node

The Agent Node is a core component of the Sequential Agent architecture. It acts as a decision-maker and orchestrator within our workflow.

Understanding the Agent Node

When the Agent Node receives input from preceding nodes, it includes the full conversation history state.messages, including the current user query, along with any custom State that has been set up. Using this input and its defined "persona" (established by the System Prompt), the Agent Node determines if external tools are necessary to fulfill the user's request.

• If tools are required, the Agent Node autonomously selects and executes the appropriate tool. This execution can be automatic or, for sensitive tasks, require human approval (HITL) before proceeding. Once the tool completes its operation, the Agent Node receives the results, processes them using the designated Chat Model (LLM), and generates a comprehensive response.

• In cases where no tools are needed, the Agent Node directly leverages the Chat Model (LLM) to formulate a response based on the current conversation context.

Inputs

	Required	Description
External Tools	Yes	Provides the Agent Node with access to a suite of external tools, enabling it to perform actions and retrieve information.
Chat Model	No	Add a new Chat Model to overwrite the default Chat Model (LLM) of the workflow. Only compatible with models that are capable of function calling.
Start Node	Yes	Receives the initial user input, along with the custom State (if set up) and the rest of the default state.messages array from the Start Node.
Agent Node	Yes	Receives input from a preceding Agent Node, enabling chained agent actions and maintaining conversational context
LLM Node	Yes	Receives the output from LLM Node, enabling the Agent Node to process the LLM's response.
Tool Node	Yes	Receives the output from a Tool Node, enabling the Agent Node to process and integrate tool's outputs into its response.

The Agent Node requires at least one connection from the following nodes: Start Node, Agent Node, LLM Node, or Tool Node.

Outputs

The Agent Node can connect to the following nodes as outputs:

• Agent Node: Passes control to a subsequent Agent Node, enabling the chaining of multiple agent actions within a workflow. This allows for more complex conversational flows and task orchestration.

• LLM Node: Passes the agent's output to an LLM Node, enabling further language processing, response generation, or decision-making based on the agent's actions and insights.

• Condition Agent Node: Directs the flow to a Condition Agent Node. This node evaluates the Agent Node's output and its predefined conditions to determine the appropriate next step in the workflow.

• Condition Node: Similar to the Condition Agent Node, the Condition Node uses predefined conditions to assess the Agent Node's output, directing the flow along different branches based on the outcome.

• End Node: Concludes the conversation flow. This occurs when the agent has successfully addressed the user's request or determined that no further action is required.

• Loop Node: Redirects the flow back to a previous node, enabling iterative or cyclical processes within the workflow. This is useful for tasks that require multiple steps or involve refining results based on previous interactions. For example, you might loop back to an earlier Agent Node or LLM Node to gather additional information or refine the conversation flow based on the current Agent Node's output.

Node Setup

Parameter	Required	Description
Agent Name	Yes	Add a descriptive name to the Agent Node to enhance workflow readability and easily target it back when using loops within the workflow.
System Prompt	No	Defines the agent's 'persona' and guides its behavior. For example, "You are a customer service agent specializing in technical support [...]."
Require Approval	No	Activates the Human-in-the-loop (HITL) feature. If set to 'True,' the Agent Node will request human approval before executing any tool. This is particularly valuable for sensitive operations or when human oversight is desired. Defaults to 'False,' allowing the Agent Node to execute tools autonomously.

Additional Parameters

Parameter	Required	Description
Human Prompt	No	This prompt is appended to the state.messages array as a human message. It allows us to inject a human-like message into the conversation flow after the Agent Node has processed its input and before the next node receives the Agent Node's output.
Approval Prompt	No	A customizable prompt presented to the human reviewer when the HITL feature is active. This prompt provides context about the tool execution, including the tool's name and purpose. The variable {tools} within the prompt will be dynamically replaced with the actual list of tools suggested by the agent, ensuring the human reviewer has all necessary information to make an informed decision.
Approve Button Text	No	Customizes the text displayed on the button for approving tool execution in the HITL interface. This allows for tailoring the language to the specific context and ensuring clarity for the human reviewer.
Reject Button Text	No	Customizes the text displayed on the button for rejecting tool execution in the HITL interface. Like the Approve Button Text, this customization enhances clarity and provides a clear action for the human reviewer to take if they deem the tool execution unnecessary or potentially harmful.
Update State	No	Provides a mechanism to modify the shared custom State object within the workflow. This is useful for storing information gathered by the agent or influencing the behavior of subsequent nodes.

Best Practices

Pro Tips

Clear system prompt

Craft a concise and unambiguous System Prompt that accurately reflects the agent's role and capabilities. This guides the agent's decision-making and ensures it acts within its defined scope.

Strategic tool selection

Choose and configure the tools available to the Agent Node, ensuring they align with the agent's purpose and the overall goals of the workflow.

HITL for sensitive tasks

Utilize the 'Require Approval' option for tasks involving sensitive data, requiring human judgment, or carrying a risk of unintended consequences.

Leverage custom State updates

Update the custom State object strategically to store gathered information or influence the behavior of downstream nodes.

Potential Pitfalls

Agent inaction due to tool overload

• Problem: When an Agent Node has access to a large number of tools within a single workflow execution, it might struggle to decide which tool is the most appropriate to use, even when a tool is clearly necessary. This can lead to the agent failing to call any tool at all, resulting in incomplete or inaccurate responses.

• Example: Imagine a customer support agent designed to handle a wide range of inquiries. You've equipped it with tools for order tracking, billing information, product returns, technical support, and more. A user asks, "What's the status of my order?" but the agent, overwhelmed by the number of potential tools, responds with a generic answer like, "I can help you with that. What's your order number?" without actually using the order tracking tool.

• Solution

  1. Prioritize and categorize tools: Design your workflow and tool integration in a way that allows you to prioritize or categorize tools based on the context of the conversation. For example, you could:

     • Use Conditional Nodes earlier in the flow to narrow down the potential tool options based on user intent.

     • Organize tools into groups within your workflow, making it easier for the agent to select from a smaller, more relevant subset.

  2. Refine system prompts: Provide clearer instructions and examples within the Agent Node's System Prompt to guide it towards the correct tool selection. If needed, emphasize the specific capabilities of each tool and the situations in which they should be used.

  3. Limit tool choices per node: If possible, break down complex workflows into smaller, more manageable segments, each with a more focused set of tools. This can help reduce the cognitive load on the agent and improve its tool-selection accuracy.

Overlooking HITL for sensitive tasks

• Problem: Failing to utilize the Agent Node's "Require Approval" (HITL) feature for tasks involving sensitive information, critical decisions, or actions with potential real-world consequences can lead to unintended outcomes or damage to user trust.

• Example: Your travel booking agent has access to a user's payment information and can automatically book flights and hotels. Without HITL, a misinterpretation of user intent or an error in the agent's understanding could result in an incorrect booking or unauthorized use of the user's payment details.

• Solution

  1. Identify sensitive actions: Analyze your workflow and identify any actions that:

     • Involve accessing or processing sensitive data (e.g., payment info, personal details).

     • Have significant financial implications.

     • Could have negative real-world consequences if performed incorrectly.

  2. Implement "Require Approval": For these sensitive actions, enable the "Require Approval" option within the Agent Node. This ensures that a human reviews the agent's proposed action and the relevant context before any sensitive data is accessed or any irreversible action is taken.

  3. Design clear approval prompts: Provide clear and concise prompts for human reviewers, summarizing the agent's intent, the proposed action, and the relevant information needed for the reviewer to make an informed decision.

Unclear or incomplete system prompt

• Problem: The System Prompt provided to the Agent Node lacks the necessary specificity and context to guide the agent effectively in carrying out its intended tasks. A vague or overly general prompt can lead to irrelevant responses, difficulty in understanding user intent, and an inability to leverage tools or data appropriately.

• Example: You're building a travel booking agent, and your System Prompt simply states: "You are a helpful AI assistant." This lacks the specific instructions and context needed for the agent to effectively guide users through flight searches, hotel bookings, and itinerary planning.

• Solution: Craft a detailed and context-aware System Prompt.

```
You are a travel booking agent. Your primary goal is to assist users in planning and booking their trips. 
- Guide them through searching for flights, finding accommodations, and exploring destinations.
- Be polite, patient, and offer travel recommendations based on their preferences.
- Utilize available tools to access flight data, hotel availability, and destination information.
```

---

5. LLM Node

As the Agent Node, the LLM Node is also a core component of the Sequential Agent architecture. While both the LLM Node and Agent Node utilize the same Chat Models (LLMs) by default, providing the same basic language processing capabilities, the LLM Node distinguishes itself in these key areas.

Here's a breakdown of the key differences

• Structured data: The LLM Node provides a dedicated feature to define a JSON schema for its output. This makes it exceptionally easy to extract structured information from the LLM's responses and pass that data to Tool Nodes in the correct JSON format. The Agent Node does not have this built-in JSON schema feature.

• HITL: While both nodes support HITL for tool execution, the LLM Node defers this control to the Tool Node itself, providing more flexibility in workflow design.

Choosing the Right Node: LLM Node vs. Agent Node

For a full comparison table about this, we refer you to this section. In the meantime, here's a quick overview:

• Choose the LLM Node when you need to extract structured data from LLM responses using the built-in JSON schema feature, or when we require fine-grained control over the LLM's output and its interaction with tools.

• Choose the Agent Node when you're focused on creating a more conversational flow, managing dialogue turns, and orchestrating multiple tools at a higher level.

Inputs

	Required	Description
Chat Model	No	Add a new Chat Model to overwrite the default Chat Model (LLM) of the workflow. Only compatible with models that are capable of function calling.
Start Node	Yes	Receives the initial user input, along with the custom State (if set up) and the rest of the default state.messages array from the Start Node.
Agent Node	Yes	Receives output from an Agent Node, which may include tool execution results or agent-generated responses.
LLM Node	Yes	Receives output from another LLM Node, enabling chained reasoning or information processing across multiple LLM Nodes.
Tool Node	Yes	Receives output from a Tool Node, providing the results of tool execution for further processing or response generation.

The LLM Node requires at least one connection from the following nodes: Start Node, Agent Node, LLM Node, or Tool Node.

Node Setup

Item	Required	Description
LLM Node Name	Yes	Add a descriptive name to the LLM Node to enhance workflow readability and easily target it back when using loops within the workflow.

Outputs

The LLM Node can connect to the following nodes as outputs:

• Agent Node: Passes the LLM's output to an Agent Node, which can then use the information to decide on actions, execute tools, or guide the conversation flow.

• LLM Node: Passes the output to a subsequent LLM Node, enabling chaining of multiple LLM operations. This is useful for tasks like refining text generation, performing multiple analyses, or breaking down complex language processing into stages.

• Condition Agent Node: Directs the flow to a Condition Agent Node. This node evaluates the LLM Node's output and its predefined conditions to determine the appropriate next step in the workflow.

• Condition Node: Similar to the Condition Agent Node, the Condition Node uses predefined conditions to assess the LLM Node's output, directing the flow along different branches based on the outcome.

• End Node: Concludes the conversation flow. This occurs when the LLM Node has successfully addressed the user's request or determined that no further action is required.

• Loop Node: Redirects the flow back to a previous node, enabling iterative or cyclical processes within the workflow. This could be used to refine the LLM's output over multiple iterations.

Additional Parameters

Parameter	Required	Description
System Prompt	No	Defines the agent's 'persona' and guides its behavior. For example, "You are a customer service agent specializing in technical support [...]."
Human Prompt	No	This prompt is appended to the state.messages array as a human message. It allows us to inject a human-like message into the conversation flow after the LLM Node has processed its input and before the next node receives the LLM Node's output.
JSON Structured Output	No	To instruct the LLM (Chat Model) to provide the output in JSON structure schema (Key, Type, Enum Values, Description).
Update State	No	Provides a mechanism to modify the shared custom State object within the workflow. This is useful for storing information gathered by the LLM Node or influencing the behavior of subsequent nodes.

Best Practices

Pro Tips

Clear system prompt

Craft a concise and unambiguous System Prompt that accurately reflects the LLM Node's role and capabilities. This guides the LLM Node's decision-making and ensures it acts within its defined scope.

Optimize for structured output

Keep your JSON schemas as straightforward as possible, focusing on the essential data elements. Only enable JSON Structured Output when you need to extract specific data points from the LLM's response or when downstream nodes require JSON input.

Strategic tool selection

Choose and configure the tools available to the LLM Node (via the Tool Node), ensuring they align with the application purpose and the overall goals of the workflow.

HITL for sensitive tasks

Utilize the 'Require Approval' option for tasks involving sensitive data, requiring human judgment, or carrying a risk of unintended consequences.

Leverage State updates

Update the custom State object strategically to store gathered information or influence the behavior of downstream nodes.

Potential Pitfalls

Unintentional tool execution due to Incorrect HITL setup

• Problem: While the LLM Node can trigger Tool Nodes, it relies on the Tool Node's configuration for Human-in-the-Loop (HITL) approval. Failing to properly configure HITL for sensitive actions can lead to tools being executed without human review, potentially causing unintended consequences.

• Example: Your LLM Node is designed to interact with a tool that makes changes to user data. You intend to have a human review these changes before execution, but the connected Tool Node's "Require Approval" option is not enabled. This could result in the tool automatically modifying user data based solely on the LLM's output, without any human oversight.

• Solution

  1. Double-Check tool node settings: Always ensure that the "Require Approval" option is enabled within the settings of any Tool Node that handles sensitive actions.

  2. Test HITL thoroughly: Before deploying your workflow, test the HITL process to ensure that human review steps are triggered as expected and that the approval/rejection mechanism functions correctly.

Overuse or misunderstanding of JSON structured output

• Problem: While the LLM Node's JSON Structured Output feature is powerful, misusing it or not fully understanding its implications can lead to data errors.

• Example: You define a complex JSON schema for the LLM Node's output, even though the downstream tasks only require a simple text response. This adds unnecessary complexity and makes your workflow harder to understand and maintain. Additionally, if the LLM's output doesn't conform to the defined schema, it can cause errors in subsequent nodes.

• Solution

  1. Use JSON output strategically: Only enable JSON Structured Output when you have a clear need to extract specific data points from the LLM's response or when the downstream Tool Nodes require JSON input.

  2. Keep schemas simple: Design your JSON schemas to be as simple and concise as possible, focusing only on the data elements that are absolutely necessary for the task.

---

6. Tool Node

The Tool Node is a key component of the Sequential Agent system, enabling the integration and execution of external tools within conversational workflows. It acts as a bridge between the language-based processing of LLM Nodes and the specialized functionalities of external tools, APIs, or services.

Understanding the Tool Node

The Tool Node's primary function is to execute external tools based on instructions received from an LLM Node and to provide flexibility for Human-in-the-Loop (HITL) intervention in the tool execution process. Here is how it works:

1. Tool Call Reception: The Tool Node receives input from an LLM Node. If the LLM's output contains the tool_calls property, the Tool Node will proceed with tool execution.

2. Execution: The Tool Node directly passes the LLM's tool_calls (which include the tool name and any required parameters) to the specified external tool. Otherwise, the Tool Node does not execute any tools in that particular workflow execution. It does not process or interpret the LLM's output in any way.

3. Human-in-the-Loop (HITL): The Tool Node allows for optional HITL, enabling human review and approval or rejection of tool execution before it occurs.

4. Output passing: After the tool execution (either automatic or after HITL approval), the Tool Node receives the tool's output and passes it to the next node in the workflow.

Inputs

	Required	Description
LLM Node	Yes	Receives the output from an LLM Node, which may or may not contain tool_calls property. If it is present, the Tool Node will use them to execute the specified tool.
External Tools	No	Provides the Tool Node with access to a suite of external tools, enabling it to perform actions and retrieve information.

Node Setup

Item	Required	Description
Name of the Tool Node	Yes	Add a descriptive name to the Tool Node to enhance workflow readability.
Require Approval (HITL)	No	Activates the Human-in-the-loop (HITL) feature. If set to 'True,' the Tool Node will request human approval before executing any tool. This is particularly valuable for sensitive operations or when human oversight is desired. Defaults to 'False,' allowing the Tool Node to execute tools autonomously.

Outputs

The Tool Node can connect to the following nodes as outputs:

• Agent Node: Passes the Tool Node's output (the result of the executed tool) to an Agent Node. The Agent Node can then use this information to decide on actions, execute further tools, or guide the conversation flow.

• LLM Node: Passes the output to a subsequent LLM Node. This enables the integration of tool results into the LLM's processing, allowing for further analysis or refinement of the conversation flow based on the tool's output.

• Condition Agent Node: Directs the flow to a Condition tool Node. This node evaluates the Tool Node's output and its predefined conditions to determine the appropriate next step in the workflow.

• Condition Node: Similar to the Condition Agent Node, the Condition Node uses predefined conditions to assess the Tool Node's output, directing the flow along different branches based on the outcome.

• End Node: Concludes the conversation flow. This might occur if the Tool Node has successfully executed its task and no further processing or interaction is needed.

• Loop Node: Redirects the flow back to a previous node, enabling iterative or cyclical processes within the workflow. This could be used for tasks that require multiple tool executions or involve refining the conversation based on tool results.

Additional Parameters

Parameter	Required	Description
Approval Prompt	No	A customizable prompt presented to the human reviewer when the HITL feature is active. This prompt provides context about the tool execution, including the tool's name and purpose. The variable {tools} within the prompt will be dynamically replaced with the actual list of tools suggested by the LLM Node, ensuring the human reviewer has all necessary information to make an informed decision.
Approve Button Text	No	Customizes the text displayed on the button for approving tool execution in the HITL interface. This allows for tailoring the language to the specific context and ensuring clarity for the human reviewer.
Reject Button Text	No	Customizes the text displayed on the button for rejecting tool execution in the HITL interface. Like the Approve Button Text, this customization enhances clarity and provides a clear action for the human reviewer to take if they deem the tool execution unnecessary or potentially harmful.
Update State	No	Provides a mechanism to modify the custom State object within the workflow. This is useful for storing information gathered by the Tool Node (after the tool execution) or influencing the behavior of subsequent nodes.

Best Practices

Pro Tips

Strategic HITL placement

Consider which tools require human oversight (HITL) and enable the "Require Approval" option accordingly.

Informative Approval Prompts

When using HITL, design clear and informative prompts for human reviewers. Provide sufficient context from the conversation and summarize the tool's intended action.

Potential Pitfalls

Unhandled tool output formats

• Problem: The Tool Node outputs data in a format that is not expected or handled by subsequent nodes in the workflow, leading to errors or incorrect processing.

• Example: A Tool Node retrieves data from an API in XML format, but the following LLM Node expects JSON input, causing a parsing error.

• Solution: Ensure that the output format of the external tool is compatible with the input requirements of the nodes connected to the Tool Node's output. Use JavaScript code within the workflow to transform or parse the tool's output as needed.

---

7. Conditional Node

The Conditional Node acts as a decision-making point in Sequential Agent workflows, evaluating a set of predefined conditions to determine the flow's next path.

Understanding the Conditional Node

The Conditional Node is essential for building workflows that adapt to different situations and user inputs. It examines the current State of the conversation, which includes all messages exchanged and any custom State variables previously defined. Then, based on the evaluation of the conditions specified in the node setup, the Conditional Node directs the flow to one of its outputs.

For instance, after an Agent or LLM Node provides a response, a Conditional Node could check if the response contains a specific keyword or if a certain condition is met in the custom State. If it does, the flow might be directed to an Agent Node for further action. If not, it could lead to a different path, perhaps ending the conversation or prompting the user with additional questions.

This enables us to create branches in our workflow, where the path taken depends on the data flowing through the system.

Here's how it works

• The Conditional Node receives input from any preceding node: Start Node, Agent Node, LLM Node, or Tool Node.

• It has access to the full conversation history and the custom State (if any), giving it plenty of context to work with.

• We define a condition that the node will evaluate. This could be checking for keywords, comparing values in the state, or any other logic we could implement via JavaScript.

• Based on whether the condition evaluates to true or false, the Conditional Node sends the flow down one of its predefined output paths. This creates a "fork in the road" or branch for our workflow.

How to set up conditions

The Conditional Node allows us to define dynamic branching logic in our workflow by choosing either a table-based interface or a JavaScript code editor to define the conditions that will control the conversation flow.

Conditions using CODE

The Conditional Node uses JavaScript to evaluate specific conditions within the conversation flow.

We can set up conditions based on keywords, State changes, or other factors to dynamically guide the workflow to different branches based on the context of the conversation. Here are some examples:

Keyword condition

This checks if a specific word or phrase exists in the conversation history.

• Example: We want to check if the user said "yes" in their last message.

• JavaScript Code

```javascript
const lastMessage = $flow.state.messages[$flow.state.messages.length - 1].content; 
return lastMessage.includes("yes") ? "Output 1" : "Output 2";
```

1. This code gets the last message from state.messages and checks if it contains "yes".
2. If "yes" is found, the flow goes to "Output 1"; otherwise, it goes to "Output 2".

State change condition

This checks if a specific value in the custom State has changed to a desired value.

• Example: We're tracking an orderStatus variable our custom State, and we want to check if it has become "confirmed".

• JavaScript Code

  return $flow.state.orderStatus === "confirmed" ? "Output 1" : "Output 2";

  1. This code directly compares the orderStatus value in our custom State to "confirmed".

  2. If it matches, the flow goes to "Output 1"; otherwise, it goes to "Output 2".

Conditions using TABLE

The Conditional Node allows us to define conditions using a user-friendly table interface, making it easy to create dynamic workflows without writing JavaScript code.

You can set up conditions based on keywords, State changes, or other factors to guide the conversation flow along different branches. Here are some examples:

Keyword condition

This checks if a specific word or phrase exists in the conversation history.

• Example: We want to check if the user said "yes" in their last message.

• Table Setup:

  Variable	Operation	Value	Output Name
  $flow.state.messages[-1].content	Is	Yes	Output 1

  1. This table entry checks if the content (.content) of the last message ([-1]) in state.messages is equal to "Yes".

  2. If the condition is met, the flow goes to "Output 1". Otherwise, the workflow is directed to a default "End" output.

State change condition

This checks if a specific value in our custom State has changed to a desired value.

• Example: We're tracking an orderStatus variable in our custom State, and we want to check if it has become "confirmed".

• Table Setup:

  Variable	Operation	Value	Output Name
  $flow.state.orderStatus	Is	Confirmed	Output 1

  1. This table entry checks if the value of orderStatus in the custom State is equal to "confirmed".

  2. If the condition is met, the flow goes to "Output 1". Otherwise, the workflow is directed to a default "End" output.

Inputs

	Required	Description
Start Node	Yes	Receives the State from the Start Node. This allows the Conditional Node to evaluate conditions based on the initial context of the conversation, including any custom State.
Agent Node	Yes	Receives the Agent Node's output. This enables the Conditional Node to make decisions based on the agent's actions and the conversation history, including any custom State.
LLM Node	Yes	Receives the LLM Node's output. This allows the Conditional Node to evaluate conditions based on the LLM's response and the conversation history, including any custom State.
Tool Node	Yes	Receives the Tool Node's output. This enables the Conditional Node to make decisions based on the results of tool execution and the conversation history, including any custom State.

The Conditional Node requires at least one connection from the following nodes: Start Node, Agent Node, LLM Node, or Tool Node.

Outputs

The Conditional Node dynamically determines its output path based on the predefined conditions, using either the table-based interface or JavaScript. This provides flexibility in directing the workflow based on condition evaluations.

Condition evaluation logic

• Table-Based conditions: The conditions in the table are evaluated sequentially, from top to bottom. The first condition that evaluates to true triggers its corresponding output. If none of the predefined conditions are met, the workflow is directed to the default "End" output.

• Code-Based conditions: When using JavaScript, we must explicitly return the name of the desired output path, including a name for the default "End" output.

• Single output path: Only one output path is activated at a time. Even if multiple conditions could be true, only the first matching condition determines the flow.

Connecting outputs

Each predefined output, including the default "End" output, can be connected to any of the following nodes:

• Agent Node: To continue the conversation with an agent, potentially taking actions based on the condition's outcome.

• LLM Node: To process the current State and conversation history with an LLM, generating responses or making further decisions.

• End Node: To terminate the conversation flow. If any output, including the default "End" output, is connected to an End Node, the Conditional Node will output the last response from the preceding node and end the workflow.

• Loop Node: To redirect the flow back to a previous sequential node, enabling iterative processes based on the condition's outcome.

Node Setup

Parameter	Required	Description
Name	No	An optional, human-readable name for the condition being evaluated. This is helpful for understanding the workflow at a glance.
Condition	Yes	This is where we define the logic that will be evaluated to determine the output paths.

Best Practices

Pro Tips

Clear condition naming

Use descriptive names for your conditions (e.g., "If user is under 18, then Policy Advisor Agent", "If order is confirmed, then End Node") to make your workflow easier to understand and debug.

Prioritize simple conditions

Start with simple conditions and gradually add complexity as needed. This makes your workflow more manageable and reduces the risk of errors.

Test thoroughly

Test your conditions with different inputs and scenarios to ensure they work as expected and cover all potential cases.

Potential Pitfalls

Mismatched condition logic and workflow design

• Problem: The conditions you define in the Conditional Node do not accurately reflect the intended logic of your workflow, leading to unexpected branching or incorrect execution paths.

• Example: You set up a condition to check if the user's age is greater than 18, but the output path for that condition leads to a section designed for users under 18.

• Solution: Review your conditions and ensure that the output paths associated with each condition match the intended workflow logic. Use clear and descriptive names for your outputs to avoid confusion.

Insufficient State management

• Problem: The Conditional Node relies on a custom state variable that is not updated correctly, leading to inaccurate condition evaluations and incorrect branching.

• Example: You're tracking a "userLocation" variable in the custom State, but the variable is not updated when the user provides their location. The Conditional Node evaluates the condition based on the outdated value, leading to an incorrect path.

• Solution: Ensure that any custom state variables used in your conditions are updated correctly throughout the workflow. Use Agent Nodes or other mechanisms to update the customk State based on user input or actions.

---

8. Conditional Agent Node

The Conditional Agent Node provides dynamic and intelligent routing within Sequential Agent flows. It combines the capabilities of the LLM Node (LLM and JSON Structured Output) and the Condition Node (user-defined conditions), allowing us to leverage agent-based reasoning and conditional logic within a single node.

Key functionalities

• Unified agent-based routing: Combines agent reasoning, structured output, and conditional logic in a single node, simplifying workflow design.

• Contextual awareness: The agent considers the entire conversation history and any custom State when evaluating conditions.

• Flexibility: Provides both table-based and code-based options for defining conditions, when catering to different user preferences and skill levels.

Setting up the Conditional Agent Node

The Conditional Agent Node acts as a specialized agent that can both process information and make routing decisions. Here's how to configure it:

1. Define the agent's persona

   • In the "System Prompt" field, provide a clear and concise description of the agent's role and the task it needs to perform for conditional routing. This prompt will guide the agent's understanding of the conversation and its decision-making process.

2. Structure the Agent's Output (Optional)

   • If you want the agent to produce structured output, use the "JSON Structured Output" feature. Define the desired schema for the output, specifying the keys, data types, and any enum values. This structured output will be used by the agent when evaluating conditions.

3. Define conditions

   • Choose either the table-based interface or the JavaScript code editor to define the conditions that will determine the routing behavior.

     • Table-Based interface: Add rows to the table, specifying the variable to check, the comparison operation, the value to compare against, and the output name to follow if the condition is met.

     • JavaScript code: Write custom JavaScript snippets to evaluate conditions. Use the return statement to specify the name of the output path to follow based on the condition's result.

4. Connect outputs

   • Connect each predefined output, including the default "End" output, to the appropriate subsequent node in the workflow. This could be an Agent Node, LLM Node, Loop Node, or an End Node.

How to set up conditions

The Conditional Agent Node allows us to define dynamic branching logic in our workflow by choose either a table-based interface or a JavaScript code editor to define the conditions that will control the conversation flow.

Conditions using CODE

The Conditional Agent Node, like the Conditional Node, uses JavaScript code to evaluate specific conditions within the conversation flow.

However, the Conditional Agent Node can evaluate conditions based on a wider range of factors, including keywords, state changes, and the content of its own output (either as free-form text or structured JSON data). This allows for more nuanced and context-aware routing decisions. Here are some examples:

Keyword condition

This checks if a specific word or phrase exists in the conversation history.

• Example: We want to check if the user said "yes" in their last message.

• JavaScript Code

```javascript
const lastMessage = $flow.state.messages[$flow.state.messages.length - 1].content; 
return lastMessage.includes("yes") ? "Output 1" : "Output 2";
```

1. This code gets the last message from `state.messages` and checks if it contains "yes".
2. If "yes" is found, the flow goes to "Output 1"; otherwise, it goes to "Output 2".

State change condition

This checks if a specific value in the custom State has changed to a desired value.

• Example: We're tracking an orderStatus variable our custom State, and we want to check if it has become "confirmed".

• JavaScript Code

  return $flow.state.orderStatus === "confirmed" ? "Output 1" : "Output 2";

  1. This code directly compares the orderStatus value in our custom State to "confirmed".

  2. If it matches, the flow goes to "Output 1"; otherwise, it goes to "Output 2".

Conditions using TABLE

The Conditional Agent Node also provides a user-friendly table interface for defining conditions, similar to the Conditional Node. You can set up conditions based on keywords, state changes, or the agent's own output, allowing you to create dynamic workflows without writing JavaScript code.

This table-based approach simplifies condition management and makes it easier to visualize the branching logic. Here are some examples:

Keyword condition

This checks if a specific word or phrase exists in the conversation history.

• Example: We want to check if the user said "yes" in their last message.

• Table Setup:

  Variable	Operation	Value	Output Name
  $flow.state.messages[-1].content	Is	Yes	Output 1

  1. This table entry checks if the content (.content) of the last message ([-1]) in state.messages is equal to "Yes".

  2. If the condition is met, the flow goes to "Output 1". Otherwise, the workflow is directed to a default "End" output.

State change condition

This checks if a specific value in our custom State has changed to a desired value.

• Example: We're tracking an orderStatus variable in our custom State, and we want to check if it has become "confirmed".

• Table Setup:

  Variable	Operation	Value	Output Name
  $flow.state.orderStatus	Is	Confirmed	Output 1

  1. This table entry checks if the value of orderStatus in the custom State is equal to "confirmed".

  2. If the condition is met, the flow goes to "Output 1". Otherwise, the workflow is directed to a default "End" output.

Input

	Required	Description
Start Node	Yes	Receives the State from the Start Node. This allows the Conditional Agent Node to evaluate conditions based on the initial context of the conversation, including any custom State.
Agent Node	Yes	Receives the Agent Node's output. This enables the Conditional Agent Node to make decisions based on the agent's actions and the conversation history, including any custom State.
LLM Node	Yes	Receives LLM Node's output. This allows the Conditional Agent Node to evaluate conditions based on the LLM's response and the conversation history, including any custom State.
Tool Node	Yes	Receives the Tool Node's output. This enables the Conditional Agent Node to make decisions based on the results of tool execution and the conversation history, including any custom State.

The Conditional Node requires at least one connection from the following nodes: Start Node, Agent Node, LLM Node, or Tool Node.

Node Setup

Parameter	Required	Description
Name	No	Add a descriptive name to the Conditional Agent Node to enhance workflow readability and easily.
Condition	Yes	This is where we define the logic that will be evaluated to determine the output paths.

Outputs

The Conditional Agent Node, like the Condition Node, dynamically determines its output path based on the conditions defined, using either the table-based interface or JavaScript. This provides flexibility in directing the workflow based on condition evaluations.

Condition evaluation logic

• Table-Based conditions: The conditions in the table are evaluated sequentially, from top to bottom. The first condition that evaluates to true triggers its corresponding output. If none of the predefined conditions are met, the workflow is directed to the default "End" output.

• Code-Based conditions: When using JavaScript, we must explicitly return the name of the desired output path, including a name for the default "End" output.

• Single output path: Only one output path is activated at a time. Even if multiple conditions could be true, only the first matching condition determines the flow.

Connecting outputs

Each predefined output, including the default "End" output, can be connected to any of the following nodes:

• Agent Node: To continue the conversation with an agent, potentially taking actions based on the condition's outcome.

• LLM Node: To process the current State and conversation history with an LLM, generating responses or making further decisions.

• End Node: To terminate the conversation flow. If the default "End" output is connected to an End Node, the Conditional Node will output the last response from the preceding node and end the conversation.

• Loop Node: To redirect the flow back to a previous sequential node, enabling iterative processes based on the condition's outcome.

Key differences from the Condition Node

• The Conditional Agent Node incorporates an agent's reasoning and structured output into the condition evaluation process.

• It provides a more integrated approach to agent-based conditional routing.

Additional Parameters

Parameter	Required	Description
System Prompt	No	Defines the Conditional Agent's 'persona' and guides its behavior for making routing decisions. For example: "You are a customer service agent specializing in technical support. Your goal is to help customers with technical issues related to our product. Based on the user's query, identify the specific technical issue (e.g., connectivity problems, software bugs, hardware malfunctions)."
Human Prompt	No	This prompt is appended to the state.messages array as a human message. It allows us to inject a human-like message into the conversation flow after the Conditional Agent Node has processed its input and before the next node receives the Conditional Agent Node's output.
JSON Structured Output	No	To instruct the Conditional Agent Node to provide the output in JSON structure schema (Key, Type, Enum Values, Description).

Best Practices

Pro Tips

Craft a clear and focused system prompt

Provide a well-defined persona and clear instructions to the agent in the System Prompt. This will guide its reasoning and help it generate relevant output for the conditional logic.

Structure output for reliable conditions

Use the JSON Structured Output feature to define a schema for the Conditional Agent's output. This will ensure that the output is consistent and easily parsable, making it more reliable for use in conditional evaluations.

Test conditions thoroughly

Test your Conditional Agent Node with various inputs and scenarios to ensure that the agent's reasoning and the conditional logic work as expected.

Potential Pitfalls

Unreliable routing due to unstructured output

• Problem: The Conditional Agent Node is not configured to output structured JSON data, leading to unpredictable output formats that can make it difficult to define reliable conditions.

• Example: The Conditional Agent Node is asked to determine user sentiment (positive, negative, neutral) but outputs its assessment as a free-form text string. The variability in the agent's language makes it challenging to create accurate conditions in the conditional table or code.

• Solution: Use the JSON Structured Output feature to define a schema for the agent's output. For example, specify a "sentiment" key with an enum of "positive," "negative," and "neutral." This will ensure that the agent's output is consistently structured, making it much easier to create reliable conditions.

---

9. Loop Node

The Loop Node allows us to create loops within our conversational flow, redirecting the conversation back to a specific point. This is useful for scenarios where we need to repeat a certain sequence of actions or questions based on user input or specific conditions.

Understanding the Loop Node

The Loop Node acts as a connector, redirecting the flow back to a specific point in the graph, allowing us to create loops within our conversational flow. It passes the current State, which includes the output of the node preceding the Loop Node to our target node. This data transfer allows our target node to process information from the previous iteration of the loop and adjust its behavior accordingly.

For instance, let's say we're building a chatbot that helps users book flights. We might use a loop to iteratively refine the search criteria based on user feedback.

Here's how the Loop Node could be used:

1. LLM Node (Initial Search): The LLM Node receives the user's initial flight request (e.g., "Find flights from Madrid to New York in July"). It queries a flight search API and returns a list of possible flights.

2. Agent Node (Present Options): The Agent Node presents the flight options to the user and asks if they would like to refine their search (e.g., "Would you like to filter by price, airline, or departure time?").

3. Conditional Agent Node: The Conditional Node checks the user's response:

   • If the user wants to refine: The flow goes to an LLM Node ("Refine Search").

   • If the user is happy with the results: The flow proceeds to the booking process.

4. LLM Node (Refine Search): This LLM Node gathers the user's refinement criteria (e.g., "Show me only flights under $500") and updates the State with the new search parameters.

5. Loop Node: The Loop Node redirects the flow back to the initial LLM Node ("Initial Search"). It passes the updated State, which now includes the refined search criteria.

6. Iteration: The initial LLM Node performs a new search using the refined criteria, and the process repeats from step 2.

In this example, the Loop Node enables an iterative search refinement process. The system can continue to loop back and refine the search results until the user is satisfied with the options presented.

The Loop Node's ability to pass the updated State back to the target node ensures that each iteration builds upon the previous one, making the search more precise and efficient.

Inputs

	Required	Description
Agent Node	Yes	Receives the output of a preceding Agent Node. This data is then sent back to the target node specified in the "Loop To" parameter.
LLM Node	Yes	Receives the output of a preceding LLM Node. This data is then sent back to the target node.
Tool Node	Yes	Receives the output of a preceding Tool Node. This data is then sent back to the target node.
Conditional Node	Yes	Receives the output of a preceding Conditional Node, indicating which branch was taken. This data is then sent back to the target node.
Conditional Agent Node	Yes	Receives the output of a preceding Conditional Agent Node. This data is then sent back to the target node.

The Loop Node requires at least one connection from the following nodes: Agent Node, LLM Node, Conditional Node, Conditional Agent Node or Tool Node.

Node Setup

Item	Required	Description
Loop To	Yes	The Loop Node requires us to specify the target node ("Loop To") where the conversational flow should be redirected. This target node must be an Agent Node or LLM Node.

Outputs

The Loop Node does not have any direct output connections. It redirects the flow back to the specific sequential node in the graph.

Best Practices

Pro Tips

Clear loop purpose

Define a clear purpose for each loop in your workflow. If possible, document with the sticky note node what you're trying to achieve with the loop.

Potencial Pitfalls

Confusing workflow structure

• Problem: Excessive or poorly designed loops make the workflow difficult to understand and maintain.

• Example: You use multiple nested loops without clear purpose or labels, making it hard to follow the flow of the conversation.

• Solution: Use loops sparingly and only when necessary. Clearly document your Loop Nodes and the nodes they connect to. Consider breaking down complex loops into smaller, more manageable units.

Infinite loops due to missing or incorrect exit conditions

• Problem: The loop never terminates because the condition that should trigger the loop's exit is either missing or incorrectly defined.

• Example: A Loop Node is used to iteratively gather user information. However, the workflow lacks a Conditional Agent Node to check if all required information has been collected. As a result, the loop continues indefinitely, repeatedly asking the user for the same information.

• Solution: Always define clear and accurate exit conditions for loops. Use Conditional Nodes to check state variables, user input, or other factors that indicate when the loop should terminate.

---

10. End Node

The End Node marks the definitive termination point of the conversation in a Sequential Agent workflow. It signifies that no further processing, actions, or interactions are required.

Understanding the End Node

The End Node serves as a signal within a Sequential Agent architecture, indicating that the conversation has reached its intended conclusion. Upon reaching the End Node, the system "understands" that the conversational objective has been met, and no further actions or interactions are required within the flow.

Input

Input	Required	Description
Agent Node	Yes	Receives the final output from a preceding Agent Node, indicating the end of the agent's interaction.
LLM Node	Yes	Receives the final output from a preceding LLM Node, marking the end of the LLM's processing.
Tool Node	Yes	Receives the final output from a preceding Tool Node, signaling the completion of the tool's execution.

The End Node requires at least one connection from the following nodes: Agent Node, LLM Node, or Tool Node.

Outputs

The End Node does not have any output connections as it signifies the termination of the information flow.

Best Practices

Pro Tips

Provide a final response

If appropriate, connect the End Node to an dedicated LLM or Agent Node to generate a final message or summary for the user, providing closure to the conversation.

Potencial Pitfalls

Premature conversation termination

• Problem: The End Node is placed too early in the workflow, causing the conversation to end before all necessary steps are completed or the user's request is fully addressed.

• Example: A chatbot designed to collect user feedback ends the conversation after the user provides their first comment, without giving them an opportunity to provide additional feedback or ask questions.

• Solution: Review your workflow logic and ensure that the End Node is placed only after all essential steps have been completed or the user has explicitly indicated their intent to end the conversation.

Lack of closure for the user

• Problem: The conversation ends abruptly without a clear signal to the user or a final message that provides a sense of closure.

• Example: A customer support chatbot ends the conversation immediately after resolving an issue, without confirming the resolution with the user or offering further assistance.

• Solution: Connect the End Node to a dedicate LLM or Agent Node to generate a final response that summarizes the conversation, confirms any actions taken, and provides a sense of closure for the user.

---

Conditional Node vs. Conditional Agent Node

Within a Sequential Agent AI architecture, both the Conditional Node and the Conditional Agent Node play an important role in creating intelligent and dynamic conversational experiences.

These nodes help us to build workflows that adapt to user input, context, and complex decision-making requirements.

While both nodes offer dynamic branching logic, they differ in their approach to condition evaluation and the level of sophistication they provide.

Conditional Node

Purpose

To create branches based on simple, predefined logical conditions.

Condition evaluation

Uses a table-based interface or JavaScript code editor to define conditions that are checked against the custom state and the full conversation history.

Output behavior

• Supports multiple output paths, each associated with a specific condition.

• Conditions are evaluated in order. The first matching condition determines the output.

• If no conditions are met, the flow follows a default "End" output.

Best suited for

• Straightforward routing decisions based on easily definable conditions.

• Workflows where the logic can be expressed using simple comparisons, keyword checks, or state variable values.

Conditional Agent Node

Purpose

To create dynamic routing based on an agent's analysis of the conversation and its structured output.

Condition evaluation

• If no Chat Model is connected, it uses the default system LLM (from the Start Node) to process the conversation history and any custom State.

• It can generate structured output, which is then used for condition evaluation.

• Uses a table-based interface or JavaScript code editor to define conditions that are checked against the agent's own output, structured or not.

Output behavior

Same as the Condition Node:

• Supports multiple output paths, each associated with a specific condition.

• Conditions are evaluated in order. The first matching condition determines the output.

• If no conditions are met, the flow follows the default "End" output.

Best suited for

• More complex routing decisions that require an understanding of conversation context, user intent, or nuanced factors.

• Scenarios where simple logical conditions are insufficient to capture the desired routing logic.

• Example: A chatbot needs to determine if a user's question is related to a specific product category. A Conditional Agent Node could be used to analyze the user's query and output a JSON object with a "category" field. The Condition Node can then use this structured output to route the user to the appropriate product specialist.

Summarizing

Feature	Conditional Node	Conditional Agent Node
Decision Logic	Based on predefined logical conditions	Based on agent's reasoning and structured output
Agent Involvement	No agent involved in condition evaluation	Uses an agent to process context and generate output for conditions
Structured Output	Not possible	Possible and encouraged for reliable condition evaluation
Condition Evaluation	Only define conditions that are checked against the full conversation history	Can define conditions that are checked against the agent's own output, structured or not
Complexity	Suitable for simple branching logic	Handles more nuanced and context-aware routing
Example use case	Routing based on user's age or a keyword in the conversation	Routing based on user sentiment, intent, or complex contextual factors

Choosing the right node

• Conditional Node: Use the Conditional Node when your routing logic involves straightforward decisions based on easily definable conditions. For instance, it's perfect for checking for specific keywords, comparing values in the State, or evaluating other simple logical expressions.

• Conditional Agent Node: However, when your routing demands a deeper understanding of the conversation's nuances, the Conditional Agent Node is the better choice. This node acts as your intelligent routing assistant, leveraging an LLM to analyze the conversation, make judgments based on context, and provide structured output that drives more sophisticated and dynamic routing.

---

Agent Node vs. LLM Node

It's important to understand that both the LLM Node and the Agent Node can be considered agentic entities within our system, as they both leverage the capabilities of a large language model (LLM) or Chat Model.

However, while both nodes can process language and interact with tools, they are designed for different purposes within a workflow. Understanding their unique capabilities is very important for building effective conversational systems.

Agent Node

Focus

The primary focus of the Agent Node to simulate the actions and decision-making of a human agent within a conversational context.

It acts as a high-level coordinator within the workflow, bringing together language understanding, tool execution, and decision-making to create a more human-like conversational experience.

Strengths

• Effectively manages the execution of multiple tools and integrates their results.

• Offers built-in support for Human-in-the-Loop (HITL), enabling human review and approval for sensitive operations.

Best Suited For

• Workflows where the agent needs to guide the user, gather information, make choices, and manage the overall conversation flow.

• Scenarios requiring integration with multiple external tools.

• Tasks involving sensitive data or actions where human oversight is beneficial, like approving financial transaction

LLM Node

Focus

Similar to the Agent Node, but it provides more flexibility when using tools and Human-in-the-Loop (HITL), both via the Tool Node.

Strengths

• Enables the definition of JSON schemas to structure the LLM's output, making it easier to extract specific information.

• Offers flexibility in tool integration, allowing for more complex sequences of LLM and tool calls, and providing fine-grained control over the HITL feature.

Best Suited For

• Scenarios where structured data needs to be extracted from the LLM's response.

• Workflows requiring a mix of automated and human-reviewed tool executions. For example, an LLM Node might call a tool to retrieve product information (automated), and then a different tool to process a payment, which would require HITL approval.

Summarizing

Feature	Agent Node	LLM Node
Tool Interaction	Directly calls and manages multiple tools, built-in HITL	Triggers tools via the Tool Node, granular HITL control at the tool level
Human-in-the-Loop (HITL)	HITL controlled at the Agent Node level (all connected tools affected)	HITL managed at the individual Tool Node level (more flexibility)
Structured Output	Relies on the LLM's natural output format	Relies on the LLM's natural output format, but, if needed, provides JSON schema definition to structure LLM output
Ideal Use Cases	Workflows with complex tool orchestration Simplified HITL at the Agent Level	Extracting structured data from LLM output Workflows with complex LLM and tool interactions, requiring mixed HITL levels

Choosing the right node

• Choose the Agent Node: Use the Agent Node when you need to create a conversational system that can manage the execution of multiple tools, all of which share the same HITL setting (enabled or disabled for the entire Agent Node). The Agent Node is also well-suited for handling complex multi-step conversations where consistent agent-like behavior is desired.

• Choose the LLM Node: On the other hand, use the LLM Node when you need to extract structured data from the LLM's output using the JSON schema feature, a capability not available in the Agent Node. The LLM Node also excels at orchestrating tool execution with fine-grained control over HITL at the individual tool level, allowing you to mix automated and human-reviewed tool executions within the same workflow by using multiple Tool Nodes connected to the LLM Node.