Class LabeledPairwiseStringEvalChain

constructor

• new LabeledPairwiseStringEvalChain(fields): LabeledPairwiseStringEvalChain

• Parameters

  • fields: LLMChainInput<EvalOutputType, BaseLanguageModelInterface<any, BaseLanguageModelCallOptions>>

  Returns LabeledPairwiseStringEvalChain

llm

outputKey

outputParser

prompt

requiresInput

skipReferenceWarning

Optional criterionName

Optional evaluationName

Optional llmKwargs

Optional memory

Optional skipInputWarning

inputKeys

• get inputKeys(): string[]

• Returns string[]

outputKeys

• get outputKeys(): string[]

• Returns string[]

apply

• apply(inputs, config?): Promise<ChainValues[]>

• Parameters

  • inputs: ChainValues[]

  • Optional config: (RunnableConfig | CallbackManager | (BaseCallbackHandler | BaseCallbackHandlerMethodsClass)[])[]

  Returns Promise<ChainValues[]>

  ⚠️ Deprecated ⚠️

  Use .batch() instead. Will be removed in 0.2.0.

  This feature is deprecated and will be removed in the future.

  It is not recommended for use.

  Call the chain on all inputs in the list

call

• call(values, config?): Promise<ChainValues>

• Run the core logic of this chain and add to output if desired.

  Wraps _call and handles memory.

  Parameters

  • values: any

  • Optional config: BaseCallbackConfig | CallbackManager | (BaseCallbackHandler | BaseCallbackHandlerMethodsClass)[]

  Returns Promise<ChainValues>

checkEvaluationArgs

• checkEvaluationArgs(reference?, input?): void

• Check if the evaluation arguments are valid.

  Parameters

  • Optional reference: string

    The reference label.

  • Optional input: string

    The input string.

  Returns void

  Throws

  If the evaluator requires an input string but none is provided, or if the evaluator requires a reference label but none is provided.

evaluateStringPairs

• evaluateStringPairs(args, callOptions?, config?): Promise<ChainValues>

• Evaluate the output string pairs.

  Parameters

  • args: LLMPairwiseStringEvaluatorArgs

  • Optional callOptions: BaseLanguageModelCallOptions

  • Optional config: BaseCallbackConfig | CallbackManager | (BaseCallbackHandler | BaseCallbackHandlerMethodsClass)[]

  Returns Promise<ChainValues>

  A dictionary containing the preference, scores, and/or other information.

invoke

• invoke(input, options?): Promise<ChainValues>

• Invoke the chain with the provided input and returns the output.

  Parameters

  • input: ChainValues

    Input values for the chain run.

  • Optional options: RunnableConfig

  Returns Promise<ChainValues>

  Promise that resolves with the output of the chain run.

predict

• predict(values, callbackManager?): Promise<EvalOutputType>

• Format prompt with values and pass to LLM

  Parameters

  • values: any

    keys to pass to prompt template

  • Optional callbackManager: CallbackManager

    CallbackManager to use

  Returns Promise<EvalOutputType>

  Completion from LLM.

  Example

  llm.predict({ adjective: "funny" })
  Copy

prepOutputs

• prepOutputs(inputs, outputs, returnOnlyOutputs?): Promise<Record<string, unknown>>

• Parameters

  • inputs: Record<string, unknown>

  • outputs: Record<string, unknown>

  • returnOnlyOutputs: boolean = false

  Returns Promise<Record<string, unknown>>

run

• run(input, config?): Promise<string>

• Parameters

  • input: any

  • Optional config: RunnableConfig | CallbackManager | (BaseCallbackHandler | BaseCallbackHandlerMethodsClass)[]

  Returns Promise<string>

  Deprecated

  Use .invoke() instead. Will be removed in 0.2.0.

serialize

• serialize(): SerializedLLMChain

• Returns SerializedLLMChain

  Deprecated

Static deserialize

• deserialize(data): Promise<LLMChain<string, BaseLanguageModel<any, BaseLanguageModelCallOptions>>>

• Parameters

  • data: SerializedLLMChain

  Returns Promise<LLMChain<string, BaseLanguageModel<any, BaseLanguageModelCallOptions>>>

Static fromLLM

• fromLLM(llm, criteria?, chainOptions?): Promise<PairwiseStringEvalChain>

• Create a new instance of the PairwiseStringEvalChain.

  Parameters

  • llm: BaseLanguageModelInterface<any, BaseLanguageModelCallOptions>

  • Optional criteria: "detail" | ConstitutionalPrinciple | {
        [key: string]: string;
    } | "conciseness" | "relevance" | "correctness" | "coherence" | "harmfulness" | "maliciousness" | "helpfulness" | "controversiality" | "misogyny" | "criminality" | "insensitivity" | "depth" | "creativity"

    The criteria to use for evaluation.

  • Optional chainOptions: Partial<Omit<LLMEvalChainInput<EvalOutputType, BaseLanguageModelInterface<any, BaseLanguageModelCallOptions>>, "llm">>

    Options to pass to the chain.

  Returns Promise<PairwiseStringEvalChain>

Static resolvePairwiseCriteria

• resolvePairwiseCriteria(criteria?): Record<string, string>

• Parameters

  • Optional criteria: "detail" | ConstitutionalPrinciple | {
        [key: string]: string;
    } | "conciseness" | "relevance" | "correctness" | "coherence" | "harmfulness" | "maliciousness" | "helpfulness" | "controversiality" | "misogyny" | "criminality" | "insensitivity" | "depth" | "creativity"

  Returns Record<string, string>