Skip to content

ModelClient

autogen.ModelClient #

Bases: Protocol

A client class must implement the following methods: - create must return a response object that implements the ModelClientResponseProtocol - cost must return the cost of the response - get_usage must return a dict with the following keys: - prompt_tokens - completion_tokens - total_tokens - cost - model

This class is used to create a client that can be used by OpenAIWrapper. The response returned from create must adhere to the ModelClientResponseProtocol but can be extended however needed. The message_retrieval method must be implemented to return a list of str or a list of messages from the response.

RESPONSE_USAGE_KEYS class-attribute instance-attribute #

RESPONSE_USAGE_KEYS = ['prompt_tokens', 'completion_tokens', 'total_tokens', 'cost', 'model']

ModelClientResponseProtocol #

Bases: Protocol

choices instance-attribute #

choices

model instance-attribute #

model

Choice #

Bases: Protocol

message instance-attribute #
message
Message #

Bases: Protocol

content instance-attribute #
content

create #

create(params)
Source code in autogen/oai/client.py 
268
def create(self, params: dict[str, Any]) -> ModelClientResponseProtocol: ...  # pragma: no cover

message_retrieval #

message_retrieval(response)

Retrieve and return a list of strings or a list of Choice.Message from the response.

NOTE: if a list of Choice.Message is returned, it currently needs to contain the fields of OpenAI's ChatCompletion Message object, since that is expected for function or tool calling in the rest of the codebase at the moment, unless a custom agent is being used.

Source code in autogen/oai/client.py 
270
271
272
273
274
275
276
277
278
def message_retrieval(
    self, response: ModelClientResponseProtocol
) -> Union[list[str], list[ModelClient.ModelClientResponseProtocol.Choice.Message]]:
    """Retrieve and return a list of strings or a list of Choice.Message from the response.

    NOTE: if a list of Choice.Message is returned, it currently needs to contain the fields of OpenAI's ChatCompletion Message object,
    since that is expected for function or tool calling in the rest of the codebase at the moment, unless a custom agent is being used.
    """
    ...  # pragma: no cover

cost #

cost(response)
Source code in autogen/oai/client.py 
280
def cost(self, response: ModelClientResponseProtocol) -> float: ...  # pragma: no cover

get_usage staticmethod #

get_usage(response)

Return usage summary of the response using RESPONSE_USAGE_KEYS.

Source code in autogen/oai/client.py 
282
283
284
285
@staticmethod
def get_usage(response: ModelClientResponseProtocol) -> dict:
    """Return usage summary of the response using RESPONSE_USAGE_KEYS."""
    ...  # pragma: no cover