Completions

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

The prompt(s) to generate completions for, encoded as array of strings or a string.

Name of the model to be used, can either be the name of a LoRA or a base model.

How many completions to generate for each prompt.

Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.

Whether to allow the model to continue generating tokens after the stop sequence. If true, the model will continue generating tokens until it reaches the max_tokens limit.

The maximum number of tokens that can be generated in the chat completion.

Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.

Whether to stream back partial progress.

What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. We generally recommend altering this or top_p but not both.

An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. We generally recommend altering this or temperature but not both.

A list of chat completion choices. Can of more than one if the n parameter is greater than 1.

The id of the request.

The model used to generate the completions.

The usage statistics for the completion request.