Thoth handles interactions between the Workbench and OpenAI's servers in support of Workbench's MercuryAI features.
A Completion represents a single interaction via OpenAI's API. It records the details of the request and response with some metadata for bookkeeping.
Completions are required to be flagged with an associted project, which is normally the Workbench project name. An optional user string indicates which user requested the operation that resulted in the completion.
A competion has a status field which defaults to "pending". A status of "error" indicates an error from the OpenAI interface, either it directly reports an error in the response or the network reported a communication error. A status of "ready" means the completion contains a successful response to the prompt contained in the request.
Completion objects may be marked as hidden, which removes them from most database queries (please note, to access hidden records, queries must start from Completion.unscoped).
The typical process for running a Completion is to create the database instance, fill in the request field with the appropriate JSON, and execute its complete! method. This runs the interaction against the OpenAI server and stores the response in the response field, setting the status field to "error" or "ready", as appropriate.
Any AI task is assumed to deliver its results in a single, primary Completion. This completion should be marked with the deliverable property set to true. If a task requires more than one OpenAI interaction, the additional Completion objects should have their deliverable flag set to false, and the corresponding deliverable Completion should include a metadata key (see below) of parent_ids, containing an array with the ids of the associated, non-deliverable Completions.
Completion objects track associated metadata. API requests to the thoth server may include a user_metadata parameter which is always directly copied (unchanged) into the deliverable Completion's metadata field, under the user_metadata key. This is meant to allow clients to associate data with the AI task that the analysis doesn't require to perform the task.
Each analysis model has an associated name and an optional version. These are automatically copied to the metadata field under the model and model_version keys, respectively.
The remaining metadata keys are for use by the analysis model to track any properties its designer deems useful.
Thoth's API exposes a concept called an "analysis model". Each model represents a single AI-related task, which may result in one or more deliverable completion (and potentially more non-deliverable completions). The main API accepts the name of an analysis model and the associated parameters, and executes the model.
AnalysisInteractorModel implementations normally use a mixin module, called AnalysisInteractor. This is patterned after the Interactor class from the ruby interactor gem. The main difference is that the operation is separated into two phases, called prepare and continue. The prepare phase is executed immediately upon invoking the interactor, but the continue phase is executed separately, by way of the AnalysisInteractorJob, using active job.
As with the original Interactor, the mixin provides hooks for support code. Where the original provides before and after, the custom version provides before_prepare (aliased as before), after_prepare, before_continue, and after_continue (aliased as after) to reflect its two-phase structure.
Invoking the interactor with call will return the Context object that results from the prepare phase only. When the continue phase is run, this same context is available to the interactor, but any further changes made to it in the continue phase are not available to the requester.
The prepare phase should be used to plan the task. The resulting context should have a deliverable property with the deliverable completion associated with the task. This completion is normally in pending state, and may or may not have a valid OpenAI request object.
The actual execution of any completion objects should be deferred to the continue phase of the interactor.
Analysis interactors are also free to use instance variables to transfer data between the prepare and continue phases. Any instance variables set by the prepare phase will be saved before starting the AnalysisInteractorJob and restored before it executes the continue phase. One restriction is that the values stored in these instance variables must be supported by ActiveJob. In practice, this means database records and the primitive types, but support has also been added to permit instances of Template to be saved.
The system currently maintains several support classes that help writing analysis models. By including ModelBase, the model will automatically define a metadata instance variable, initially populated with the model, model version, and user metadata. The contents of this instance variable are then used to update the deliverable completion's metadata after each stage of the interactor.
The ModelBase provides helper methods to correctly build OpenAI request objects. The class method openai_model sets the OpenAI model to use (it currently defaults to "gpt-4"), system_prompt sets the system prompt (used with "chat" completions), and token_limit (default 1250) sets the number of tokens that OpenAI may use for its response. Prompts must reserve space for this many tokens.
If the values of these settings are not constants, then instance methods with similar names may be defined to compute the correct value from the context and instance variables.
It provides a method request(prompt) that takes a prompt string and returns an OpenAI request structure appropriate for the chosen model. Most interactors should find this enough to create correctly structured requests.
[To be continued...]
When generating new prompts, use these checklists:
All prompts generated by the viewer:
All Live Labs analyses:
session_typeshared_session_text to write the introductionAll Topline analyses: