Create tests for langchain plugins

Create tests for langchain plugins on verify_plugins/verify_langchain.py similar to the tests under verify_plugins/verify_plugnplai.py using the langchain documentation below:

AI Plugin Example:

1. Langchain ChatGPT Plugins

This example shows how to use ChatGPT Plugins within LangChain abstractions.

Note 1: This currently only works for plugins with no auth.

Note 2: There are almost certainly other ways to do this, this is just a first pass. If you have better ideas, please open a PR!

from langchain.chat_models import ChatOpenAI
from langchain.agents import load_tools, initialize_agent
from langchain.agents import AgentType
from langchain.tools import AIPluginTool

tool = AIPluginTool.from_plugin_url("https://www.klarna.com/.well-known/ai-plugin.json")

llm = ChatOpenAI(temperature=0)
tools = load_tools(["requests_all"])
tools += [tool]

agent_chain = initialize_agent(
tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True
)
agent_chain.run("what t shirts are available in klarna?")

> Entering new AgentExecutor chain...
I need to check the Klarna Shopping API to see if it has information on available t shirts.
Action: KlarnaProducts
Action Input: None
Observation: Usage Guide: Use the Klarna plugin to get relevant product suggestions for any shopping or researching purpose. The query to be sent should not include stopwords like articles, prepositions and determinants. The api works best when searching for words that are related to products, like their name, brand, model or category. Links will always be returned and should be shown to the user.

OpenAPI Spec: {'openapi': '3.0.1', 'info': {'version': 'v0', 'title': 'Open AI Klarna product Api'}, 'servers': [{'url': 'https://www.klarna.com/us/shopping'}], 'tags': [{'name': 'open-ai-product-endpoint', 'description': 'Open AI Product Endpoint. Query for products.'}], 'paths': {'/public/openai/v0/products': {'get': {'tags': ['open-ai-product-endpoint'], 'summary': 'API for fetching Klarna product information', 'operationId': 'productsUsingGET', 'parameters': [{'name': 'q', 'in': 'query', 'description': 'query, must be between 2 and 100 characters', 'required': True, 'schema': {'type': 'string'}}, {'name': 'size', 'in': 'query', 'description': 'number of products returned', 'required': False, 'schema': {'type': 'integer'}}, {'name': 'budget', 'in': 'query', 'description': 'maximum price of the matching product in local currency, filters results', 'required': False, 'schema': {'type': 'integer'}}], 'responses': {'200': {'description': 'Products found', 'content': {'application/json': {'schema': {'$ref': '#/components/schemas/ProductResponse'}}}}, '503': {'description': 'one or more services are unavailable'}}, 'deprecated': False}}}, 'components': {'schemas': {'Product': {'type': 'object', 'properties': {'attributes': {'type': 'array', 'items': {'type': 'string'}}, 'name': {'type': 'string'}, 'price': {'type': 'string'}, 'url': {'type': 'string'}}, 'title': 'Product'}, 'ProductResponse': {'type': 'object', 'properties': {'products': {'type': 'array', 'items': {'$ref': '#/components/schemas/Product'}}}, 'title': 'ProductResponse'}}}}
Thought:I need to use the Klarna Shopping API to search for t shirts.
Action: requests_get
Action Input: https://www.klarna.com/us/shopping/public/openai/v0/products?q=t%20shirts
Observation: {"products":[{"name":"Lacoste Men's Pack of Plain T-Shirts","url":"https://www.klarna.com/us/shopping/pl/cl10001/3202043025/Clothing/Lacoste-Men-s-Pack-of-Plain-T-Shirts/?utm_source=openai","price":"$26.60","attributes":["Material:Cotton","Target Group:Man","Color:White,Black"]},{"name":"Hanes Men's Ultimate 6pk. Crewneck T-Shirts","url":"https://www.klarna.com/us/shopping/pl/cl10001/3201808270/Clothing/Hanes-Men-s-Ultimate-6pk.-Crewneck-T-Shirts/?utm_source=openai","price":"$13.82","attributes":["Material:Cotton","Target Group:Man","Color:White"]},{"name":"Nike Boy's Jordan Stretch T-shirts","url":"https://www.klarna.com/us/shopping/pl/cl359/3201863202/Children-s-Clothing/Nike-Boy-s-Jordan-Stretch-T-shirts/?utm_source=openai","price":"$14.99","attributes":["Material:Cotton","Color:White,Green","Model:Boy","Size (Small-Large):S,XL,L,M"]},{"name":"Polo Classic Fit Cotton V-Neck T-Shirts 3-Pack","url":"https://www.klarna.com/us/shopping/pl/cl10001/3203028500/Clothing/Polo-Classic-Fit-Cotton-V-Neck-T-Shirts-3-Pack/?utm_source=openai","price":"$29.95","attributes":["Material:Cotton","Target Group:Man","Color:White,Blue,Black"]},{"name":"adidas Comfort T-shirts Men's 3-pack","url":"https://www.klarna.com/us/shopping/pl/cl10001/3202640533/Clothing/adidas-Comfort-T-shirts-Men-s-3-pack/?utm_source=openai","price":"$14.99","attributes":["Material:Cotton","Target Group:Man","Color:White,Black","Neckline:Round"]}]}
Thought:The available t shirts in Klarna are Lacoste Men's Pack of Plain T-Shirts, Hanes Men's Ultimate 6pk. Crewneck T-Shirts, Nike Boy's Jordan Stretch T-shirts, Polo Classic Fit Cotton V-Neck T-Shirts 3-Pack, and adidas Comfort T-shirts Men's 3-pack.
Final Answer: The available t shirts in Klarna are Lacoste Men's Pack of Plain T-Shirts, Hanes Men's Ultimate 6pk. Crewneck T-Shirts, Nike Boy's Jordan Stretch T-shirts, Polo Classic Fit Cotton V-Neck T-Shirts 3-Pack, and adidas Comfort T-shirts Men's 3-pack.

> Finished chain.

#2 Custom Agent with PlugIns

from typing import Callable

Set up a prompt template

class CustomPromptTemplate(StringPromptTemplate):
# The template to use
template: str
############## NEW ######################
# The list of tools available
tools_getter: Callable

def format(self, **kwargs) -> str:
    # Get the intermediate steps (AgentAction, Observation tuples)
    # Format them in a particular way
    intermediate_steps = kwargs.pop("intermediate_steps")
    thoughts = ""
    for action, observation in intermediate_steps:
        thoughts += action.log
        thoughts += f"\nObservation: {observation}\nThought: "
    # Set the agent_scratchpad variable to that value
    kwargs["agent_scratchpad"] = thoughts
    ############## NEW ######################
    tools = self.tools_getter(kwargs["input"])
    # Create a tools variable from the list of tools provided
    kwargs["tools"] = "\n".join(
        [f"{tool.name}: {tool.description}" for tool in tools]
    )
    # Create a list of tool names for the tools provided
    kwargs["tool_names"] = ", ".join([tool.name for tool in tools])
    return self.template.format(**kwargs)

prompt = CustomPromptTemplate(
template=template,
tools_getter=get_tools,
# This omits the agent_scratchpad, tools, and tool_names variables because those are generated dynamically
# This includes the intermediate_steps variable because that is needed
input_variables=["input", "intermediate_steps"],
)

Output Parser
The output parser is unchanged from the previous notebook, since we are not changing anything about the output format.

class CustomOutputParser(AgentOutputParser):
def parse(self, llm_output: str) -> Union[AgentAction, AgentFinish]:
# Check if agent should finish
if "Final Answer:" in llm_output:
return AgentFinish(
# Return values is generally always a dictionary with a single output key
# It is not recommended to try anything else at the moment :)
return_values={"output": llm_output.split("Final Answer:")[-1].strip()},
log=llm_output,
)
# Parse out the action and action input
regex = r"Action\s*\d*\s*:(.?)\nAction\s\d*\sInput\s\d*\s*:[\s](.)"
match = re.search(regex, llm_output, re.DOTALL)
if not match:
raise ValueError(f"Could not parse LLM output: {llm_output}")
action = match.group(1).strip()
action_input = match.group(2)
# Return the action and action input
return AgentAction(
tool=action, tool_input=action_input.strip(" ").strip('"'), log=llm_output
)

output_parser = CustomOutputParser()

Set up LLM, stop sequence, and the agent
Also the same as the previous notebook

llm = OpenAI(temperature=0)

LLM chain consisting of the LLM and a prompt

llm_chain = LLMChain(llm=llm, prompt=prompt)

tool_names = [tool.name for tool in tools]
agent = LLMSingleActionAgent(
llm_chain=llm_chain,
output_parser=output_parser,
stop=["\nObservation:"],
allowed_tools=tool_names,
)

Use the Agent
Now we can use it!

agent_executor = AgentExecutor.from_agent_and_tools(
agent=agent, tools=tools, verbose=True
)

agent_executor.run("what shirts can i buy?")

> Entering new AgentExecutor chain...
Thought: I need to find a product API
Action: Open_AI_Klarna_product_Api.productsUsingGET
Action Input: shirts

Observation:I found 10 shirts from the API response. They range in price from $9.99 to $450.00 and come in a variety of materials, colors, and patterns. I now know what shirts I can buy
Final Answer: Arg, I found 10 shirts from the API response. They range in price from $9.99 to $450.00 and come in a variety of materials, colors, and patterns.

> Finished chain.





'Arg, I found 10 shirts from the API response. They range in price from $9.99 to $450.00 and come in a variety of materials, colors, and patterns.'

3. Reference:

classlangchain.tools.AIPluginTool(*, name, description, args_schema=<class 'langchain.tools.plugin.AIPluginToolSchema'>, return_direct=False, verbose=False, callbacks=None, callback_manager=None, handle_tool_error=False, plugin, api_spec)[source]
Bases: langchain.tools.base.BaseTool

Parameters
name (str) –

description (str) –

args_schema (Type[langchain.tools.plugin.AIPluginToolSchema]) –

return_direct (bool) –

verbose (bool) –

callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) –

callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) –

handle_tool_error (Optional[Union[bool, str, Callable[[langchain.tools.base.ToolException], str]]]) –

plugin (langchain.tools.plugin.AIPlugin) –

api_spec (str) –

Return type
None

attributeapi_spec: str[Required]
attributeargs_schema: Type[AIPluginToolSchema]= <class 'langchain.tools.plugin.AIPluginToolSchema'>
Pydantic model class to validate and parse the tool’s input arguments.

attributeplugin: AIPlugin[Required]
classmethodfrom_plugin_url(url)[source]
Parameters
url (str) –

Return type
langchain.tools.plugin.AIPluginTool

classlangchain.tools.APIOperation(*, operation_id, description=None, base_url, path, method, properties, request_body=None)[source]
Bases: pydantic.main.BaseModel

A model for a single API operation.

Parameters
operation_id (str) –

description (Optional[str]) –

base_url (str) –

path (str) –

method (langchain.tools.openapi.utils.openapi_utils.HTTPVerb) –

properties (Sequence[langchain.tools.openapi.utils.api_models.APIProperty]) –

request_body (Optional[langchain.tools.openapi.utils.api_models.APIRequestBody]) –

Return type
None

attributebase_url: str[Required]
The base URL of the operation.

attributedescription: Optional[str]= None
The description of the operation.

attributemethod: langchain.tools.openapi.utils.openapi_utils.HTTPVerb[Required]
The HTTP method of the operation.

attributeoperation_id: str[Required]
The unique identifier of the operation.

attributepath: str[Required]
The path of the operation.

attributeproperties: Sequence[langchain.tools.openapi.utils.api_models.APIProperty][Required]
attributerequest_body: Optional[langchain.tools.openapi.utils.api_models.APIRequestBody]= None
The request body of the operation.

classmethodfrom_openapi_spec(spec, path, method)[source]
Create an APIOperation from an OpenAPI spec.

Parameters
spec (langchain.tools.openapi.utils.openapi_utils.OpenAPISpec) –

path (str) –

method (str) –

Return type
langchain.tools.openapi.utils.api_models.APIOperation

classmethodfrom_openapi_url(spec_url, path, method)[source]
Create an APIOperation from an OpenAPI URL.

Parameters
spec_url (str) –

path (str) –

method (str) –

Return type
langchain.tools.openapi.utils.api_models.APIOperation

to_typescript()[source]
Get typescript string representation of the operation.

Return type
str

staticts_type_from_python(type_)[source]
Parameters
type_ (Union[str, Type, tuple, None, enum.Enum]) –

Return type
str

propertybody_params: List[str]

propertypath_params: List[str]

propertyquery_params: List[str]

Hey @edreisMD,

I've started working on this PR. The plan is to create a new file verify_langchain.py under the verify_plugins directory to contain our tests for the langchain plugins. These tests will be based on the examples you provided and will check the functionality of the AIPluginTool and APIOperation classes when used with plugin URLs and OpenAPI specs or URLs.

Give me a minute!

Some code snippets I looked at (click to expand). If some file is missing from here, you can mention the path in the ticket description.

plugnplai/docs/examples/get-list-of-plugins.md

Lines 1 to 20 in 66c4988

    
           # Get list of plugins 
        
           Here is an example of how to get a list of plugins from the [plugnplai.com](https://plugnplai.com) API: 
        
           ```python 
        
           import plugnplai 
        
           # Get all plugins from plugnplai.com 
        
           urls = plugnplai.get_plugins() 
        
           #  Get ChatGPT plugins - only ChatGPT verified plugins 
        
           urls = plugnplai.get_plugins(filter = 'ChatGPT') 
        
           #  Get working plugins - only tested plugins (in progress) 
        
           urls = plugnplai.get_plugins(filter = 'working') 
        
           # Get the Manifest and the OpenAPI specification from the plugin URL  
        
           manifest, openapi_spec = plugnplai.spec_from_url(urls[0]) 
        
           ```

plugnplai/plugnplai/utils.py

Lines 45 to 73 in 66c4988

    
               """ 
        
               if provider == "plugnplai": 
        
                   base_url = "https://www.plugnplai.com/_functions" 
        
                   # Construct the endpoint URL based on the filter and category arguments 
        
                   if filter in ["working", "ChatGPT"]: 
        
                       url = f'{base_url.strip("/")}/getUrls/{filter}' 
        
                   elif verified_for in ["langchain", "plugnplai"]: 
        
                       url = f'{base_url.strip("/")}/getUrls/{verified_for}' 
        
                   elif category is not None: 
        
                       url = f'{base_url.strip("/")}/getCategoryUrls/{category}' 
        
                   else: 
        
                       url = f'{base_url.strip("/")}/getUrls' 
        
                   # Make the HTTP GET request 
        
                   response = make_request_get(url) 
        
                   # Check if the response status code is successful (200 OK) 
        
                   if response.status_code == 200: 
        
                       # Parse the JSON response and return the result 
        
                       return response.json() 
        
                   else: 
        
                       # Handle unsuccessful responses 
        
                       return f"An error occurred: {response.status_code} {response.reason}" 
        
               elif provider == "pluginso": 
        
                   url = "https://plugin.so/api/plugins/list" 
        
                   response = make_request_get(url) 
        
                   if response.status_code == 200: 
        
                       # Parse the JSON response and return the result 
        
                       return [f"https://{entry['domain']}" for entry in response.json()] 
        
                   else: 
        
                       # Handle unsuccessful responses

plugnplai/examples/apply_plugins_three_steps.ipynb

Lines 52 to 86 in 66c4988

    
           }, 
        
           { 
        
             "cell_type": "code", 
        
             "execution_count": 13, 
        
             "metadata": {}, 
        
             "outputs": [], 
        
             "source": [ 
        
               "import plugnplai as pl\n", 
        
               "from langchain.chat_models import ChatOpenAI\n", 
        
               "from langchain.prompts.chat import ChatPromptTemplate, SystemMessagePromptTemplate, AIMessagePromptTemplate, HumanMessagePromptTemplate\n", 
        
               "from langchain.schema import AIMessage, HumanMessage, SystemMessage\n", 
        
               "from IPython.display import display, Markdown" 
        
             ] 
        
           }, 
        
           { 
        
             "cell_type": "markdown", 
        
             "metadata": { 
        
               "id": "7fYY9i4Z3sBO" 
        
             }, 
        
             "source": [ 
        
               "# 1. Load plugins from [plugnplai.com](https://plugnplai.com)" 
        
             ] 
        
           }, 
        
           { 
        
             "cell_type": "markdown", 
        
             "metadata": { 
        
               "id": "nlC-9wyp2vtF" 
        
             }, 
        
             "source": [ 
        
               "Lets find one plugin for each category, using PlugnPlai categories (see [API reference](https://plugnplai.github.io/))\n", 
        
               "- travel\n", 
        
               "- shopping \n", 
        
               "- weather" 
        
             ] 
        
           },

plugnplai/docs/examples/apply_plugins_three_steps.ipynb

Lines 52 to 86 in 66c4988

    
           }, 
        
           { 
        
             "cell_type": "code", 
        
             "execution_count": 13, 
        
             "metadata": {}, 
        
             "outputs": [], 
        
             "source": [ 
        
               "import plugnplai as pl\n", 
        
               "from langchain.chat_models import ChatOpenAI\n", 
        
               "from langchain.prompts.chat import ChatPromptTemplate, SystemMessagePromptTemplate, AIMessagePromptTemplate, HumanMessagePromptTemplate\n", 
        
               "from langchain.schema import AIMessage, HumanMessage, SystemMessage\n", 
        
               "from IPython.display import display, Markdown" 
        
             ] 
        
           }, 
        
           { 
        
             "cell_type": "markdown", 
        
             "metadata": { 
        
               "id": "7fYY9i4Z3sBO" 
        
             }, 
        
             "source": [ 
        
               "## 1. Load plugins from [plugnplai.com](https://plugnplai.com)" 
        
             ] 
        
           }, 
        
           { 
        
             "cell_type": "markdown", 
        
             "metadata": { 
        
               "id": "nlC-9wyp2vtF" 
        
             }, 
        
             "source": [ 
        
               "Lets find one plugin for each category, using PlugnPlai categories (see [API reference](https://plugnplai.github.io/))\n", 
        
               "- travel\n", 
        
               "- shopping \n", 
        
               "- weather" 
        
             ] 
        
           },

plugnplai/README.md

Lines 1 to 26 in 66c4988

    
           # 🎸 Plug and Plai 
        
           Plug and Plai is an open source library aiming to simplify the integration of AI plugins into open-source language models (LLMs). 
        
           It provides utility functions to get a list of active plugins from [plugnplai.com](https://plugnplai.com/) directory, get plugin manifests, and extract OpenAPI specifications and load plugins. 
        
           ## Installation 
        
           You can install Plug and PlAI using pip: 
        
           ```python 
        
           pip install plugnplai 
        
           ``` 
        
           ## Quick Start Example 
        
           **Load and Call Step by Step:** [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/edreisMD/plugnplai/blob/main/examples/plugins_step_by_step.ipynb) 
        
           ## More Examples 
        
           **Generate Prompt with Plugins Description:** [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/edreisMD/plugnplai/blob/main/examples/create_prompt_plugins.ipynb) 
        
           **Plugins Retrieval:** [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/edreisMD/plugnplai/blob/main/examples/plugin_retriever_with_langchain_agent.ipynb)

I'm a bot that handles simple bugs and feature requests but I might make mistakes. Please be kind!

I'm sorry, but it looks like an error has occured. Try removing and re-adding the sweep label. I'll try again in a minute. If this error persists contact team@sweep.dev.

I'm a bot that handles simple bugs and feature requests but I might make mistakes. Please be kind!

I'm sorry, but it looks like an error has occured. Try removing and re-adding the sweep label. I'll try again in a minute. If this error persists contact team@sweep.dev.

I'm a bot that handles simple bugs and feature requests but I might make mistakes. Please be kind!

Hey @edreisMD,

I've started working on this PR. The plan is to create a new file verify_langchain.py under the verify_plugins directory. This file will contain tests for the langchain plugins based on the examples you provided. I'll be setting up test cases and using the assert statement to check if the actual output of the functions and methods matches the expected output.

Give me a minute!

Some code snippets I looked at (click to expand). If some file is missing from here, you can mention the path in the ticket description.

plugnplai/docs/examples/get-list-of-plugins.md

Lines 1 to 20 in 66c4988

    
           # Get list of plugins 
        
           Here is an example of how to get a list of plugins from the [plugnplai.com](https://plugnplai.com) API: 
        
           ```python 
        
           import plugnplai 
        
           # Get all plugins from plugnplai.com 
        
           urls = plugnplai.get_plugins() 
        
           #  Get ChatGPT plugins - only ChatGPT verified plugins 
        
           urls = plugnplai.get_plugins(filter = 'ChatGPT') 
        
           #  Get working plugins - only tested plugins (in progress) 
        
           urls = plugnplai.get_plugins(filter = 'working') 
        
           # Get the Manifest and the OpenAPI specification from the plugin URL  
        
           manifest, openapi_spec = plugnplai.spec_from_url(urls[0]) 
        
           ```

plugnplai/plugnplai/utils.py

Lines 45 to 73 in 66c4988

    
               """ 
        
               if provider == "plugnplai": 
        
                   base_url = "https://www.plugnplai.com/_functions" 
        
                   # Construct the endpoint URL based on the filter and category arguments 
        
                   if filter in ["working", "ChatGPT"]: 
        
                       url = f'{base_url.strip("/")}/getUrls/{filter}' 
        
                   elif verified_for in ["langchain", "plugnplai"]: 
        
                       url = f'{base_url.strip("/")}/getUrls/{verified_for}' 
        
                   elif category is not None: 
        
                       url = f'{base_url.strip("/")}/getCategoryUrls/{category}' 
        
                   else: 
        
                       url = f'{base_url.strip("/")}/getUrls' 
        
                   # Make the HTTP GET request 
        
                   response = make_request_get(url) 
        
                   # Check if the response status code is successful (200 OK) 
        
                   if response.status_code == 200: 
        
                       # Parse the JSON response and return the result 
        
                       return response.json() 
        
                   else: 
        
                       # Handle unsuccessful responses 
        
                       return f"An error occurred: {response.status_code} {response.reason}" 
        
               elif provider == "pluginso": 
        
                   url = "https://plugin.so/api/plugins/list" 
        
                   response = make_request_get(url) 
        
                   if response.status_code == 200: 
        
                       # Parse the JSON response and return the result 
        
                       return [f"https://{entry['domain']}" for entry in response.json()] 
        
                   else: 
        
                       # Handle unsuccessful responses

plugnplai/docs/examples/apply_plugins_three_steps.ipynb

Lines 52 to 86 in 66c4988

    
           }, 
        
           { 
        
             "cell_type": "code", 
        
             "execution_count": 13, 
        
             "metadata": {}, 
        
             "outputs": [], 
        
             "source": [ 
        
               "import plugnplai as pl\n", 
        
               "from langchain.chat_models import ChatOpenAI\n", 
        
               "from langchain.prompts.chat import ChatPromptTemplate, SystemMessagePromptTemplate, AIMessagePromptTemplate, HumanMessagePromptTemplate\n", 
        
               "from langchain.schema import AIMessage, HumanMessage, SystemMessage\n", 
        
               "from IPython.display import display, Markdown" 
        
             ] 
        
           }, 
        
           { 
        
             "cell_type": "markdown", 
        
             "metadata": { 
        
               "id": "7fYY9i4Z3sBO" 
        
             }, 
        
             "source": [ 
        
               "## 1. Load plugins from [plugnplai.com](https://plugnplai.com)" 
        
             ] 
        
           }, 
        
           { 
        
             "cell_type": "markdown", 
        
             "metadata": { 
        
               "id": "nlC-9wyp2vtF" 
        
             }, 
        
             "source": [ 
        
               "Lets find one plugin for each category, using PlugnPlai categories (see [API reference](https://plugnplai.github.io/))\n", 
        
               "- travel\n", 
        
               "- shopping \n", 
        
               "- weather" 
        
             ] 
        
           },

plugnplai/examples/apply_plugins_three_steps.ipynb

Lines 52 to 86 in 66c4988

    
           }, 
        
           { 
        
             "cell_type": "code", 
        
             "execution_count": 13, 
        
             "metadata": {}, 
        
             "outputs": [], 
        
             "source": [ 
        
               "import plugnplai as pl\n", 
        
               "from langchain.chat_models import ChatOpenAI\n", 
        
               "from langchain.prompts.chat import ChatPromptTemplate, SystemMessagePromptTemplate, AIMessagePromptTemplate, HumanMessagePromptTemplate\n", 
        
               "from langchain.schema import AIMessage, HumanMessage, SystemMessage\n", 
        
               "from IPython.display import display, Markdown" 
        
             ] 
        
           }, 
        
           { 
        
             "cell_type": "markdown", 
        
             "metadata": { 
        
               "id": "7fYY9i4Z3sBO" 
        
             }, 
        
             "source": [ 
        
               "# 1. Load plugins from [plugnplai.com](https://plugnplai.com)" 
        
             ] 
        
           }, 
        
           { 
        
             "cell_type": "markdown", 
        
             "metadata": { 
        
               "id": "nlC-9wyp2vtF" 
        
             }, 
        
             "source": [ 
        
               "Lets find one plugin for each category, using PlugnPlai categories (see [API reference](https://plugnplai.github.io/))\n", 
        
               "- travel\n", 
        
               "- shopping \n", 
        
               "- weather" 
        
             ] 
        
           },

plugnplai/README.md

Lines 1 to 26 in 66c4988

    
           # 🎸 Plug and Plai 
        
           Plug and Plai is an open source library aiming to simplify the integration of AI plugins into open-source language models (LLMs). 
        
           It provides utility functions to get a list of active plugins from [plugnplai.com](https://plugnplai.com/) directory, get plugin manifests, and extract OpenAPI specifications and load plugins. 
        
           ## Installation 
        
           You can install Plug and PlAI using pip: 
        
           ```python 
        
           pip install plugnplai 
        
           ``` 
        
           ## Quick Start Example 
        
           **Load and Call Step by Step:** [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/edreisMD/plugnplai/blob/main/examples/plugins_step_by_step.ipynb) 
        
           ## More Examples 
        
           **Generate Prompt with Plugins Description:** [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/edreisMD/plugnplai/blob/main/examples/create_prompt_plugins.ipynb) 
        
           **Plugins Retrieval:** [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/edreisMD/plugnplai/blob/main/examples/plugin_retriever_with_langchain_agent.ipynb)

I'm a bot that handles simple bugs and feature requests but I might make mistakes. Please be kind!

Hey @edreisMD,

I've started working on this PR. The plan is to create a new file verify_plugins/verify_langchain.py for the langchain plugin tests. In this file, we'll have tests for creating AIPluginTool and APIOperation instances, running an AgentExecutor chain with AIPluginTool instances, using APIOperation instances in a custom agent, and reading plugin and OpenAPI spec files.

Give me a minute!

Some code snippets I looked at (click to expand). If some file is missing from here, you can mention the path in the ticket description.

plugnplai/docs/examples/get-list-of-plugins.md

Lines 1 to 20 in 66c4988

    
           # Get list of plugins 
        
           Here is an example of how to get a list of plugins from the [plugnplai.com](https://plugnplai.com) API: 
        
           ```python 
        
           import plugnplai 
        
           # Get all plugins from plugnplai.com 
        
           urls = plugnplai.get_plugins() 
        
           #  Get ChatGPT plugins - only ChatGPT verified plugins 
        
           urls = plugnplai.get_plugins(filter = 'ChatGPT') 
        
           #  Get working plugins - only tested plugins (in progress) 
        
           urls = plugnplai.get_plugins(filter = 'working') 
        
           # Get the Manifest and the OpenAPI specification from the plugin URL  
        
           manifest, openapi_spec = plugnplai.spec_from_url(urls[0]) 
        
           ```

plugnplai/plugnplai/utils.py

Lines 45 to 73 in 66c4988

    
               """ 
        
               if provider == "plugnplai": 
        
                   base_url = "https://www.plugnplai.com/_functions" 
        
                   # Construct the endpoint URL based on the filter and category arguments 
        
                   if filter in ["working", "ChatGPT"]: 
        
                       url = f'{base_url.strip("/")}/getUrls/{filter}' 
        
                   elif verified_for in ["langchain", "plugnplai"]: 
        
                       url = f'{base_url.strip("/")}/getUrls/{verified_for}' 
        
                   elif category is not None: 
        
                       url = f'{base_url.strip("/")}/getCategoryUrls/{category}' 
        
                   else: 
        
                       url = f'{base_url.strip("/")}/getUrls' 
        
                   # Make the HTTP GET request 
        
                   response = make_request_get(url) 
        
                   # Check if the response status code is successful (200 OK) 
        
                   if response.status_code == 200: 
        
                       # Parse the JSON response and return the result 
        
                       return response.json() 
        
                   else: 
        
                       # Handle unsuccessful responses 
        
                       return f"An error occurred: {response.status_code} {response.reason}" 
        
               elif provider == "pluginso": 
        
                   url = "https://plugin.so/api/plugins/list" 
        
                   response = make_request_get(url) 
        
                   if response.status_code == 200: 
        
                       # Parse the JSON response and return the result 
        
                       return [f"https://{entry['domain']}" for entry in response.json()] 
        
                   else: 
        
                       # Handle unsuccessful responses

plugnplai/examples/apply_plugins_three_steps.ipynb

Lines 52 to 86 in 66c4988

    
           }, 
        
           { 
        
             "cell_type": "code", 
        
             "execution_count": 13, 
        
             "metadata": {}, 
        
             "outputs": [], 
        
             "source": [ 
        
               "import plugnplai as pl\n", 
        
               "from langchain.chat_models import ChatOpenAI\n", 
        
               "from langchain.prompts.chat import ChatPromptTemplate, SystemMessagePromptTemplate, AIMessagePromptTemplate, HumanMessagePromptTemplate\n", 
        
               "from langchain.schema import AIMessage, HumanMessage, SystemMessage\n", 
        
               "from IPython.display import display, Markdown" 
        
             ] 
        
           }, 
        
           { 
        
             "cell_type": "markdown", 
        
             "metadata": { 
        
               "id": "7fYY9i4Z3sBO" 
        
             }, 
        
             "source": [ 
        
               "# 1. Load plugins from [plugnplai.com](https://plugnplai.com)" 
        
             ] 
        
           }, 
        
           { 
        
             "cell_type": "markdown", 
        
             "metadata": { 
        
               "id": "nlC-9wyp2vtF" 
        
             }, 
        
             "source": [ 
        
               "Lets find one plugin for each category, using PlugnPlai categories (see [API reference](https://plugnplai.github.io/))\n", 
        
               "- travel\n", 
        
               "- shopping \n", 
        
               "- weather" 
        
             ] 
        
           },

plugnplai/docs/examples/apply_plugins_three_steps.ipynb

Lines 52 to 86 in 66c4988

    
           }, 
        
           { 
        
             "cell_type": "code", 
        
             "execution_count": 13, 
        
             "metadata": {}, 
        
             "outputs": [], 
        
             "source": [ 
        
               "import plugnplai as pl\n", 
        
               "from langchain.chat_models import ChatOpenAI\n", 
        
               "from langchain.prompts.chat import ChatPromptTemplate, SystemMessagePromptTemplate, AIMessagePromptTemplate, HumanMessagePromptTemplate\n", 
        
               "from langchain.schema import AIMessage, HumanMessage, SystemMessage\n", 
        
               "from IPython.display import display, Markdown" 
        
             ] 
        
           }, 
        
           { 
        
             "cell_type": "markdown", 
        
             "metadata": { 
        
               "id": "7fYY9i4Z3sBO" 
        
             }, 
        
             "source": [ 
        
               "## 1. Load plugins from [plugnplai.com](https://plugnplai.com)" 
        
             ] 
        
           }, 
        
           { 
        
             "cell_type": "markdown", 
        
             "metadata": { 
        
               "id": "nlC-9wyp2vtF" 
        
             }, 
        
             "source": [ 
        
               "Lets find one plugin for each category, using PlugnPlai categories (see [API reference](https://plugnplai.github.io/))\n", 
        
               "- travel\n", 
        
               "- shopping \n", 
        
               "- weather" 
        
             ] 
        
           },

plugnplai/README.md

Lines 1 to 26 in 66c4988

    
           # 🎸 Plug and Plai 
        
           Plug and Plai is an open source library aiming to simplify the integration of AI plugins into open-source language models (LLMs). 
        
           It provides utility functions to get a list of active plugins from [plugnplai.com](https://plugnplai.com/) directory, get plugin manifests, and extract OpenAPI specifications and load plugins. 
        
           ## Installation 
        
           You can install Plug and PlAI using pip: 
        
           ```python 
        
           pip install plugnplai 
        
           ``` 
        
           ## Quick Start Example 
        
           **Load and Call Step by Step:** [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/edreisMD/plugnplai/blob/main/examples/plugins_step_by_step.ipynb) 
        
           ## More Examples 
        
           **Generate Prompt with Plugins Description:** [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/edreisMD/plugnplai/blob/main/examples/create_prompt_plugins.ipynb) 
        
           **Plugins Retrieval:** [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/edreisMD/plugnplai/blob/main/examples/plugin_retriever_with_langchain_agent.ipynb)

I'm a bot that handles simple bugs and feature requests but I might make mistakes. Please be kind!

	# Get list of plugins

	Here is an example of how to get a list of plugins from the [plugnplai.com](https://plugnplai.com) API:

	```python
	import plugnplai

	# Get all plugins from plugnplai.com
	urls = plugnplai.get_plugins()

	# Get ChatGPT plugins - only ChatGPT verified plugins
	urls = plugnplai.get_plugins(filter = 'ChatGPT')

	# Get working plugins - only tested plugins (in progress)
	urls = plugnplai.get_plugins(filter = 'working')


	# Get the Manifest and the OpenAPI specification from the plugin URL
	manifest, openapi_spec = plugnplai.spec_from_url(urls[0])
	```

	"""
	if provider == "plugnplai":
	base_url = "https://www.plugnplai.com/_functions"
	# Construct the endpoint URL based on the filter and category arguments
	if filter in ["working", "ChatGPT"]:
	url = f'{base_url.strip("/")}/getUrls/{filter}'
	elif verified_for in ["langchain", "plugnplai"]:
	url = f'{base_url.strip("/")}/getUrls/{verified_for}'
	elif category is not None:
	url = f'{base_url.strip("/")}/getCategoryUrls/{category}'
	else:
	url = f'{base_url.strip("/")}/getUrls'
	# Make the HTTP GET request
	response = make_request_get(url)
	# Check if the response status code is successful (200 OK)
	if response.status_code == 200:
	# Parse the JSON response and return the result
	return response.json()
	else:
	# Handle unsuccessful responses
	return f"An error occurred: {response.status_code} {response.reason}"
	elif provider == "pluginso":
	url = "https://plugin.so/api/plugins/list"
	response = make_request_get(url)
	if response.status_code == 200:
	# Parse the JSON response and return the result
	return [f"https://{entry['domain']}" for entry in response.json()]
	else:
	# Handle unsuccessful responses

	},
	{
	"cell_type": "code",
	"execution_count": 13,
	"metadata": {},
	"outputs": [],
	"source": [
	"import plugnplai as pl\n",
	"from langchain.chat_models import ChatOpenAI\n",
	"from langchain.prompts.chat import ChatPromptTemplate, SystemMessagePromptTemplate, AIMessagePromptTemplate, HumanMessagePromptTemplate\n",
	"from langchain.schema import AIMessage, HumanMessage, SystemMessage\n",
	"from IPython.display import display, Markdown"
	]
	},
	{
	"cell_type": "markdown",
	"metadata": {
	"id": "7fYY9i4Z3sBO"
	},
	"source": [
	"# 1. Load plugins from [plugnplai.com](https://plugnplai.com)"
	]
	},
	{
	"cell_type": "markdown",
	"metadata": {
	"id": "nlC-9wyp2vtF"
	},
	"source": [
	"Lets find one plugin for each category, using PlugnPlai categories (see [API reference](https://plugnplai.github.io/))\n",
	"- travel\n",
	"- shopping \n",
	"- weather"
	]
	},

	# 🎸 Plug and Plai

	Plug and Plai is an open source library aiming to simplify the integration of AI plugins into open-source language models (LLMs).

	It provides utility functions to get a list of active plugins from [plugnplai.com](https://plugnplai.com/) directory, get plugin manifests, and extract OpenAPI specifications and load plugins.

	## Installation

	You can install Plug and PlAI using pip:

	```python
	pip install plugnplai
	```

	## Quick Start Example

	Load and Call Step by Step: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/edreisMD/plugnplai/blob/main/examples/plugins_step_by_step.ipynb)


	## More Examples

	Generate Prompt with Plugins Description: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/edreisMD/plugnplai/blob/main/examples/create_prompt_plugins.ipynb)

	Plugins Retrieval: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/edreisMD/plugnplai/blob/main/examples/plugin_retriever_with_langchain_agent.ipynb)