• Fri. Nov 1st, 2024

What is Google Cloud’s generative AI evaluation service?

Byadmin

Aug 7, 2024



As part of Clarify, AWS offers enterprises a feature, dubbed FMEval, which is an open-source LLM evaluation library to help data scientists and ML engineers evaluate LLMs before deciding to use it for a specific use case.

“FMEval provides the ability to perform evaluations for both LLM model endpoints or the endpoint for a generative AI service as a whole. FMEval helps in measuring evaluation dimensions such as accuracy, robustness, bias, toxicity, and factual knowledge for any LLM,” the cloud service provider wrote in a blog post.

Enterprises can use EMEval to evaluate LLMs hosted on either AWS or third-party platforms, such as ChatGPT, HuggingFace, and LangChain, it added. 



Source link