1 Star 1 Fork 2

AI Large Model/Step Audio

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
文件
克隆/下载
call_vllm_chat.py 943 Bytes
一键复制 编辑 原始数据 按行查看 历史
mabuyun 提交于 4个月前 . add vllm docker (#112)
import argparse
from openai import OpenAI
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument(
"--server-url",
type=str,
default="http://127.0.0.1:8000",
help="VLLM server url.",
)
parser.add_argument(
"--model-name", type=str, default="step-audio-chat", help="Model name."
)
args = parser.parse_args()
server_url = args.server_url + "/v1" # for chat route
client = OpenAI(base_url=server_url, api_key="whatever")
messages = [
{
"role": "system",
"content": "You are an AI designed for conversation, currently unable to connect to the internet.",
},
{"role": "user", "content": "Introduce yourself."},
]
completion = client.chat.completions.create(
model=args.model_name,
messages=messages,
)
res = completion.choices[0].message.content
print(res)
Loading...
马建仓 AI 助手
尝试更多
代码解读
代码找茬
代码优化
1
https://gitee.com/AI-LargeModel/Step-Audio.git
git@gitee.com:AI-LargeModel/Step-Audio.git
AI-LargeModel
Step-Audio
Step Audio
main

搜索帮助