Deploy Your First LoRA

​

Sign up

To start, first please sign up for an account on Empower. By default, you will be enrolled into the free tier, giving you immediate access to explore our features and capabilities. More details about the product tiers can be found on the pricing page.

​

Check compatibility of the LoRA

Check to make sure the LoRA is compatible. Below is the list of requirements for the LoRA:

We are actively expanding our capabilities. If you have specific needs, please contact us, and we can work on the support!

• Trained using huggingface PEFT
• Hosted on huggingface
• Must be based on one of the supported base models:
  • meta-llama/Meta-Llama-3-8B-Instruct
  • meta-llama/Meta-Llama-3-70B-Instruct
  • mistralai/Mixtral-8x7B-Instruct-v0.1
• The rank(r) <= 32 Here is one of the example: empower-dev/example-lora

​

Deploy LoRA

​

Use Empower web app

Goto the lora management page and click the button “Deploy LoRA”

Fill the info into the modal:

• Hugging Face Repository ID: Hugging Face repository ID, e.g., FlagAlpha/Llama2-Chinese-7b-Chat-LoRA.
• Name: Specify the name of the repository if you want it to differ from the Hugging Face repository ID. For instance, you might want a permanent endpoint and keep the underlying LoRA transparent. Name is unique across the organization, using an existing name here will replace the existing model.
• Revision: (Optional) Revision of the Hugging Face repository is necessary if it differs from the current head.
• Hugging Face Token: (Optional) Hugging Face access token to fetch the model if the repository is private. It will only be used to fetch the model and we will not persist it.

Click deploy to deploy the LoRA and you LoRA will be ready to serve!

​

Use API

Alternatively, you can use the API to deploy LoRAs programmatically, please see the deploy a LoRA api for details.

​

Next step

Now you can query the LoRA model you’ve just deployed! Check here for details.