Query a Deployed Model

Empower provides an OpenAI-compatible API for model querying. A LoRA model can be queried immediately with negligible code startup time right after deployment. We also offer the capability to query the base model directly.

​

Create an api key

To query deployed model, you need to first have an api key, you can use the default created api or create a new one from the api key management page.

Note: All the api keys are shared across the organization.

​

Fetch the name of the model to query

​

LoRA

Name of the LoRA can be find on the lora management page.

​

Base Model

Name of the LoRA can be find on the base model management page.

​

Query the model

You then can use one of the method below to query the model:

export API_KEY= # replace with the api key
export MODEL_NAME= # replace with the model name
curl "https://app.empower.dev/api/v1/chat/completions" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $API_KEY" \
-d '{
    "model": "$MODEL_NAME",
    "messages": [
    {"role": "user", "content": "Who won the world series in 2020?"},
    {"role": "assistant", "content": "The Los Angeles Dodgers won the World Series in 2020."},
    {"role": "user", "content": "Where was it played?"}
],
    "temperature": 0.7
}'

More detail about the api can be found in the completions api.