[aiml] Lemur: Yet Another Big Model for Code

[Undescribed Horrific Abuse, One Victim & Survivor of Many]

Lemur says it is Llama-2 finetuned on code and instruct data and that
it is state of the art. It's a 70B parameter model, 70G when loaded in
8bit.

https://github.com/OpenLemur/Lemur

https://arxiv.org/abs/2310.06830
|
https://www.xlang.ai/blog/openlemur

https://huggingface.co/OpenLemur

We introduce Lemur and Lemur-Chat, openly accessible language models
optimized for both natural language and coding capabilities to serve
as the backbone of versatile language agents. The evolution from
language chat models to functional language agents demands that models
not only master human interaction, reasoning, and planning but also
ensure grounding in the relevant environments. This calls for a
harmonious blend of language and coding capabilities in the models.
Lemur and Lemur-Chat are proposed to address this necessity,
demonstrating balanced proficiencies in both domains, unlike existing
open-source models that tend to specialize in either. Through
meticulous pre-training using a code-intensive corpus and instruction
fine-tuning on text and code data, our models achieve state-of-the-art
averaged performance across diverse text and coding benchmarks among
open-source models. Comprehensive experiments demonstrate Lemur's
superiority over existing open-source models and its proficiency
across various agent tasks involving human communication, tool usage,
and interaction under fully- and partially- observable environments.
The harmonization between natural and programming languages enables
Lemur-Chat to significantly narrow the gap with proprietary models on
agent abilities, providing key insights into developing advanced
open-source agents adept at reasoning, planning, and operating
seamlessly across environments.