JosefAlbers/Roy

---
task_categories:
- question-answering
- translation
- summarization
- text-generation
- text2text-generation
- conversational
tags:
- agent
- multi-agent
- autogpt
- autogen
- agentgpt
- gptq
- wizard
- code-generation
- retrieval-augmented-generation
- humaneval
---
# [Roy: Rapid Prototyping of Agents with Hotswappable Components](https://github.com/JosefAlbers/Roy)

[<img src="https://colab.research.google.com/assets/colab-badge.svg" />](https://colab.research.google.com/github/JosefAlbers/Roy/blob/main/quickstart.ipynb)
[](https://zenodo.org/badge/latestdoi/699801819)

Roy is a lightweight alternative to `autogen` for developing advanced multi-agent systems using language models. It aims to simplify and democratize the development of emergent collective intelligence.

## Features

- **Model Agnostic**: Use any LLM, no external APIs required. Defaults to a 4-bit quantized wizard-coder-python model for efficiency.

- **Modular and Composable**: Roy decomposes agent interactions into reusable building blocks - templating, retrieving, generating, executing.

- **Transparent and Customizable**: Every method has a clear purpose. Easily swap out components or add new capabilities.

## Quickstart

```sh
git clone https://github.com/JosefAlbers/Roy
cd Roy
pip install -r requirements.txt
pip install -U transformers optimum accelerate auto-gptq --extra-index-url https://huggingface.github.io/autogptq-index/whl/cu118/
```

```python
from roy import Roy, Roys
roy = Roy()
s = '"What date is today? Which big tech stock has the largest year-to-date gain this year? How much is the gain?'
roy.generate(roy.format(s))
```

### **Rapid Benchmarking**

Roy provides a simple way to evaluate and iterate on your model architecture.. This allows you to:

- Easily swap out components, such as language models, prompt formats, agent architectures, etc

- Benchmark on different tasks like arithmetic, python coding, etc (default is OpenAI's HumanEval)

- Identify agent's areas of strengths and weaknesses

```python
from Roy.util import piecewise_human_eval

# Comparing different language models
piecewise_human_eval(0, lm_id='TheBloke/WizardCoder-Python-7B-V1.0-GPTQ')
# -> {'pass@1': 0.6341463414634146}
piecewise_human_eval(0, lm_id='TheBloke/tora-code-7B-v1.0-GPTQ')
# -> {'pass@1': 0.5609756097560976}
piecewise_human_eval(0, lm_id='TheBloke/Arithmo-Mistral-7B-GPTQ')
# -> {'pass@1': 0.5121951219512195}

# Testing a custom agent architecture
piecewise_human_eval(0, fx=<your_custom_Roy_agent>)
```

*Takes around 30 minutes each on a free Google Colab runtime.*

### **Constrained Beam Search**

Use templates to structure conversations (control output length, format, etc)

```python
roy.generate(s, ('\n```python', '\n```')) # Generate a python code block
roy.generate(s, (('\n```python', '\n```javascript'), '\n```')) # Generate python or javascript codes
roy.generate(s, ('\n```python', 100, '\n```')) # Generate a code block of size less than 100 tokens
```

### **Retrieval Augmented Generation**

Enhance generation with relevant knowledge.

```python
s = 'Create a text to image generator.'
r = roy.retrieve(s, n_topk=3, src='huggingface')
[roy.generate(s) for s in r]
```

### **Auto-Feedback**

Agents recursively improve via critiquing each other.

```python
s = "Create a secure and unique secret code word with a Python script that involves multiple steps to ensure the highest level of confidentiality and protection.\n"
for i in range(2):
c = roy.generate(s, prohibitions=['input'])
s += roy.execute(c)
```

### **Auto-Grinding**

Agents collaborate in tight loops to iteratively refine outputs to specification.

```python
user_request = "Compare the year-to-date gain for META and TESLA."
ai_response = roy.generate(user_request, ('\n```python', ' yfinance', '\n```'))
for i in range(2):
shell_execution = roy.execute(ai_response)
if 'ModuleNotFoundError' in shell_execution:
roy.execute(roy.generate(roy.format(f'Write a shell command to address the error encountered while running this Python code:\n\n{shell_execution}')))
elif 'Error' in shell_execution:
ai_response = roy.generate(roy.format(f'Modify the code to address the error encountered:\n\n{shell_execution}'))
else:
break
```

### **Multi-Agent**

Flexible primitives to build ecosystems of agents.

```python
roys = Roys()

# AutoFeedback
roys.create(agents = {'Coder': 'i = execute(generate(i))'})
roys.start(requests = {'i': 'Create a mobile application that can track the health of elderly people living alone in rural areas.'})

# Retrieval Augmented Generation
roys.create(
agents = {
'Retriever': 'r = retrieve(i)',
'Generator': 'o = generate(r)',
})
roys.start(requests = {'i': 'Create a Deutsch to English translator.'})

# Providing a custom tool to one of the agents using lambda
roys.create(
agents = {
'Coder': 'c = generate(i)',
'Proxy': 'c = custom(execute(c))',
},
tools = {'custom': lambda x:f'Modify the code to address the error encountered:\n\n{x}' if 'Error' in x else None})
roys.start(requests = {'i': 'Compare the year-to-date gain for META and TESLA.'})

# Another way to create a custom tool for agents
def custom_switch(self, c):
py_str = 'Modify the code to address the error encountered:\n\n'
sh_str = 'Write a shell command to address the error encountered while running this Python code:\n\n'
x = self.execute(c)
if 'ModuleNotFoundError' in x:
self.execute(self.generate(sh_str+x))
elif 'Error' in x:
self.dict_cache['i'] = [py_str+x]
else:
return '<<<Success>>>:\n\n'+x

roys.create(
agents = {
'Coder': 'c = generate(i)',
'Proxy': '_ = protocol(c)',
},
tools = {'protocol': custom_switch})
roys.start(requests = {'i': 'Compare the year-to-date gain for META and TESLA.'})
```

## Emergent Multi-Agent Dynamics

Roy aims to facilitate the emergence of complex, adaptive multi-agent systems. It draws inspiration from biological and AI concepts to enable decentralized coordination and continual learning.

- **Survival of the Fittest** - Periodically evaluate and selectively retain high-performing agents based on accuracy, speed etc. Agents adapt through peer interactions.

- **Mixture of Experts** - Designate agent expertise, dynamically assemble specialist teams, and route tasks to optimal experts. Continuously refine and augment experts.

These mechanisms facilitate the emergence of capable, adaptive, and efficient agent collectives.

## Get Involved

Roy is under active development. We welcome contributions - feel free to open issues and PRs!

## Support the Project

If you found this project helpful or interesting and want to support more of these experiments, feel free to buy me a coffee!

<a href="https://www.buymeacoffee.com/albersj66a" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/default-orange.png" alt="Buy Me A Coffee" height="25" width="100"></a>

原始信息汇总

Roy: 快速原型化具有可热插拔组件的代理

特性

• 模型无关性：可以使用任何大型语言模型（LLM），无需外部API。默认使用4位量化的wizard-coder-python模型以提高效率。

• 模块化和可组合性：Roy将代理交互分解为可重用的构建块，包括模板化、检索、生成和执行。

• 透明和可定制性：每个方法都有明确的目的。可以轻松替换组件或添加新功能。

快速开始

sh git clone https://github.com/JosefAlbers/Roy cd Roy pip install -r requirements.txt pip install -U transformers optimum accelerate auto-gptq --extra-index-url https://huggingface.github.io/autogptq-index/whl/cu118/

python from roy import Roy, Roys roy = Roy() s = "What date is today? Which big tech stock has the largest year-to-date gain this year? How much is the gain? roy.generate(roy.format(s))

快速基准测试

Roy提供了一种简单的方法来评估和迭代模型架构。这允许你：

• 轻松替换组件，如语言模型、提示格式、代理架构等
• 在不同的任务上进行基准测试，如算术、Python编程等（默认是OpenAI的HumanEval）
• 识别代理的优势和弱点

python from Roy.util import piecewise_human_eval

比较不同的语言模型

piecewise_human_eval(0, lm_id=TheBloke/WizardCoder-Python-7B-V1.0-GPTQ)

-> {pass@1: 0.6341463414634146}

piecewise_human_eval(0, lm_id=TheBloke/tora-code-7B-v1.0-GPTQ)

-> {pass@1: 0.5609756097560976}

piecewise_human_eval(0, lm_id=TheBloke/Arithmo-Mistral-7B-GPTQ)

-> {pass@1: 0.5121951219512195}

测试自定义代理架构

piecewise_human_eval(0, fx=<your_custom_Roy_agent>)

在免费Google Colab运行时上每次大约需要30分钟。

约束波束搜索

使用模板来结构化对话（控制输出长度、格式等）

python roy.generate(s, ( python, )) # 生成一个Python代码块 roy.generate(s, (( python, javascript), )) # 生成Python或JavaScript代码 roy.generate(s, ( python, 100, )) # 生成一个大小小于100个令牌的代码块

检索增强生成

通过相关知识增强生成。

python s = Create a text to image generator. r = roy.retrieve(s, n_topk=3, src=huggingface) [roy.generate(s) for s in r]

自动反馈

代理通过相互批评递归改进。

python s = "Create a secure and unique secret code word with a Python script that involves multiple steps to ensure the highest level of confidentiality and protection. " for i in range(2): c = roy.generate(s, prohibitions=[input]) s += roy.execute(c)

自动磨练

代理在紧密循环中协作，迭代地细化输出以符合规范。

python user_request = "Compare the year-to-date gain for META and TESLA." ai_response = roy.generate(user_request, ( python, yfinance, )) for i in range(2): shell_execution = roy.execute(ai_response) if ModuleNotFoundError in shell_execution: roy.execute(roy.generate(roy.format(fWrite a shell command to address the error encountered while running this Python code:

{shell_execution}))) elif Error in shell_execution: ai_response = roy.generate(roy.format(fModify the code to address the error encountered:

{shell_execution})) else: break

多代理

灵活的原语来构建代理生态系统。

python roys = Roys()

自动反馈

roys.create(agents = {Coder: i = execute(generate(i))}) roys.start(requests = {i: Create a mobile application that can track the health of elderly people living alone in rural areas.})

检索增强生成

roys.create( agents = { Retriever: r = retrieve(i), Generator: o = generate(r), }) roys.start(requests = {i: Create a Deutsch to English translator.})

为代理提供自定义工具使用lambda

roys.create( agents = { Coder: c = generate(i), Proxy: c = custom(execute(c)), }, tools = {custom: lambda x:fModify the code to address the error encountered:

{x} if Error in x else None}) roys.start(requests = {i: Compare the year-to-date gain for META and TESLA.})

另一种为代理创建自定义工具的方法

def custom_switch(self, c): py_str = Modify the code to address the error encountered:

sh_str = Write a shell command to address the error encountered while running this Python code:


x = self.execute(c)
if ModuleNotFoundError in x:
    self.execute(self.generate(sh_str+x))
elif Error in x:
    self.dict_cache[i] = [py_str+x]
else:
    return <<<Success>>>:

+x

roys.create( agents = { Coder: c = generate(i), Proxy: _ = protocol(c), }, tools = {protocol: custom_switch}) roys.start(requests = {i: Compare the year-to-date gain for META and TESLA.})

新兴多代理动态

Roy旨在促进复杂、自适应多代理系统的出现。它从生物学和AI概念中汲取灵感，以实现去中心化协调和持续学习。

• 适者生存：定期评估并选择性地保留高绩效代理，基于准确性、速度等。代理通过同行交互进行适应。

• 专家混合：指定代理专长，动态组建专家团队，并将任务路由到最佳专家。持续精炼和增强专家。

这些机制促进了能力、自适应和高效代理集体的出现。