StableLM: 稳定性AI的语言模型

Stability AI推出的StableLM系列包括多个不同规模和功能的模型，旨在不断优化和扩展语言处理的边界。每个模型都具备独特的架构和训练策略，以适应不同的应用需求和性能目标。

模型架构与训练详情

StableLM-3B-4E1T模型采用解码器唯一的变换器架构，与LLaMA模型类似，但进行了特定的调整以优化性能和处理能力。例如，该模型使用了旋转位置嵌入和LayerNorm标准化，以及GPT-NeoX分词器。

此外，StableLM-Alpha v2模型通过引入SwiGLU激活函数和更高质量的数据源，显著提高了模型性能。这些模型的上下文长度为4096令牌，支持复杂和细致的语言理解任务。

训练数据与效果评估

StableLM模型使用了多种开源大规模数据集，如Falcon RefinedWeb抽取和RedPajama-Data，通过这些高质量数据源确保了模型的综合性和适应性。模型的性能通过多种零样本评估任务来验证，确保其在多个标准化测试中的表现与竞争模型相当甚至更优。

如何开始使用

StableLM模型托管在Hugging Face平台，您可以通过简单的代码示例来启动并与StableLM-Tuned-Alpha模型交互，体验其独特的语言生成能力。

    
 

  
  复制
 
    
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, StoppingCriteria, StoppingCriteriaList

tokenizer = AutoTokenizer.from_pretrained("stabilityai/stablelm-tuned-alpha-7b")
model = AutoModelForCausalLM.from_pretrained("stabilityai/stablelm-tuned-alpha-7b")
model.half().cuda()

class StopOnTokens(StoppingCriteria):
    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs) -> bool:
        stop_ids = set([50278, 50279, 50277, 1, 0])
        return input_ids[0][-1] in stop_ids

system_prompt = """<|SYSTEM|># StableLM Tuned (Alpha version) - StableLM is a helpful and harmless open-source AI language model developed by StabilityAI. - StableLM is excited to be able to help the user, but will refuse to do anything that could be considered harmful to the user. - StableLM is more than just an information source, StableLM is also able to write poetry, short stories, and make jokes. - StableLM will refuse to participate in anything that could harm a human. """

prompt = f"{system_prompt}<|USER|>What's your mood today?<|ASSISTANT|>"

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
tokens = model.generate(
  **inputs,
  max_new_tokens=64,
  temperature=0.7,
  do_sample=True,
  stopping_criteria=StoppingCriteriaList([StopOnTokens()])
)
print(tokenizer.decode(tokens[0], skip_special_tokens=True))

import torch<br />
from transformers import AutoModelForCausalLM, AutoTokenizer, StoppingCriteria, StoppingCriteriaList</p>
<p>tokenizer = AutoTokenizer.from_pretrained(“stabilityai/stablelm-tuned-alpha-7b”)<br />
model = AutoModelForCausalLM.from_pretrained(“stabilityai/stablelm-tuned-alpha-7b”)<br />
model.half().cuda()</p>
<p>class StopOnTokens(StoppingCriteria):<br />
    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs) -> bool:<br />
        stop_ids = set([50278, 50279, 50277, 1, 0])<br />
        return input_ids[0][-1] in stop_ids</p>
<p>system_prompt = “””<|SYSTEM|># StableLM Tuned (Alpha version)<br />
– StableLM is a helpful and harmless open-source AI language model developed by StabilityAI.<br />
– StableLM is excited to be able to help the user, but will refuse to do anything that could be considered harmful to the user.<br />
– StableLM is more than just an information source, StableLM is also able to write poetry, short stories, and make jokes.<br />
– StableLM will refuse to participate in anything that could harm a human.<br />
“””</p>
<p>prompt = f”{system_prompt}<|USER|>What’s your mood today?<|ASSISTANT|>”</p>
<p>inputs = tokenizer(prompt, return_tensors=”pt”).to(“cuda”)<br />
tokens = model.generate(<br />
  **inputs,<br />
  max_new_tokens=64,<br />
  temperature=0.7,<br />
  do_sample=True,<br />
  stopping_criteria=StoppingCriteriaList([StopOnTokens()])<br />
)<br />
print(tokenizer.decode(tokens[0], skip_special_tokens=True))<br />

通过实际操作StableLM模型，我深刻感受到了其在处理复杂语言任务时的强大能力和灵活性。无论是进行日常对话还是生成技术内容，StableLM都表现出色。对于开发者和研究人员来说，这不仅是一个工具，更是一个探索人工智能潜能的平台。


  ×


   直达下载
  

   


反爬虫抓取，人机验证，请输入验证码查看内容
     验证码：
     提交查看
请扫描上方公众号二维码，发送关键字：2024，获取验证码。
微信搜索公众号:“AI工具教程”或者“chatgpt8080” 或微信扫描右侧二维码关注微信公众号





                    

                        温馨提示：
                    
                    

                        
文章标题：StableLM: 稳定性AI的语言模型
                        
文章链接：https://yuntunft.cn/38555.html
                        
更新时间：2024年07月05日