Llama 3 - Open Model That Is Truly Useful?

pip install -Uqqq pip --progress-bar off
pip install -qqq transformers==4.40.1 --progress-bar off
pip install -qqq accelerate==0.29.3 --progress-bar off

import os
from inspect import cleandoc
 
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from google.colab import userdata
 
os.environ["HF_TOKEN"] = userdata.get("HF_TOKEN")
 
MODEL_NAME = "meta-llama/Meta-Llama-3-8B-Instruct"
 
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
model = AutoModelForCausalLM.from_pretrained(
    MODEL_NAME,
    torch_dtype=torch.float16,
    device_map="auto",
)

messages = [
    {
        "role": "system",
        "content": "You are a professional machine learning engineer that writes in easy to understand language",
    },
    {
        "role": "user",
        "content": "What are the pros/cons of ChatGPT vs Open Source LLMs?",
    },
]
 
print(tokenizer.apply_chat_template(messages, tokenize=False))

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a professional machine learning engineer that writes in easy to
understand language<|eot_id|><|start_header_id|>user<|end_header_id|>

What are the pros/cons of ChatGPT vs Open Source LLMs?<|eot_id|>

from transformers import GenerationConfig, TextStreamer, pipeline
 
generation_config = GenerationConfig.from_pretrained(MODEL_NAME)
generation_config.max_new_tokens = 512
generation_config.temperature = 0.0001
generation_config.do_sample = True
 
streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
 
stop_token_ids = [tokenizer.eos_token_id, tokenizer.convert_tokens_to_ids("<|eot_id|>")]
 
llm = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    return_full_text=False,
    generation_config=generation_config,
    num_return_sequences=1,
    eos_token_id=stop_token_ids,
    streamer=streamer,
)

output = llm(messages)

As a machine learning engineer, I'm excited to dive into the pros and cons of
ChatGPT vs Open Source LLMs. Let's break it down!
 
**ChatGPT:**
 
Pros:
 
1. **Ease of use**: ChatGPT is a user-friendly, cloud-based AI model that
   requires minimal technical expertise to integrate into your application.
2. **Pre-trained**: ChatGPT comes pre-trained on a massive dataset, which means
   you can start using it right away without needing to train your own model.
3. **Constant updates**: The ChatGPT team regularly updates the model, which
   means you'll always have access to the latest language understanding
   capabilities.
4. **Scalability**: ChatGPT is designed to handle large volumes of
   conversations, making it suitable for high-traffic applications.
 
Cons:
 
1. **Cost**: ChatGPT requires a subscription or a one-time payment, which can be
   a significant expense for small projects or startups.
2. **Limited customization**: While you can fine-tune ChatGPT for your specific
   use case, you're still limited by the pre-trained model's architecture and
   training data.
3. **Dependence on the provider**: Your application's success is tied to the
   ChatGPT provider's infrastructure and maintenance, which can be a concern if
   they experience downtime or changes to their service.
 
**Open Source LLMs:**
 
Pros:
 
1. **Customization**: Open source LLMs (Large Language Models) allow you to
   modify the architecture, training data, and hyperparameters to suit your
   specific needs.
2. **Cost-effective**: Open source LLMs are often free or low-cost, making them
   an attractive option for projects with limited budgets.
3. **Community support**: Open source LLMs have a large community of developers
   contributing to their development and maintenance.
4. **Flexibility**: You can integrate open source LLMs with your own
   infrastructure and technology stack.
 
Cons:
 
1. **Technical expertise**: Open source LLMs require significant technical
   expertise to set up, train, and fine-tune, which can be a barrier for
   non-technical teams.
2. **Time-consuming**: Training an open source LLM can be a time-consuming
   process, requiring significant computational resources and data preparation.
3. **Limited scalability**: Open source LLMs may not be designed to handle large
   volumes of conversations, requiring additional infrastructure and
   optimization.
4. **Maintenance**: You'll need to handle maintenance, updates, and bug fixes
   for the open source LLM, which can

def create_messages(prompt: str, system_prompt: str = None):
    messages = []
    if system_prompt:
        messages.append({"role": "system", "content": cleandoc(system_prompt)})
    messages.append({"role": "user", "content": cleandoc(prompt)})
    return messages

messages = [
    {
        "role": "user",
        "content": "What is the most iconic dish that slavics prepare for Christmas?",
    }
]
 
output = llm(messages)

What a great question!
 
In many Slavic countries, Christmas is a time for traditional and hearty dishes,
often centered around meat, bread, and sweets. While there's no single most
iconic dish that all Slavic countries prepare for Christmas, I can give you some
popular examples:
 
1. **Borscht (Борщ)**: A beet-based soup that's a staple in many Slavic
   countries, including Ukraine, Russia, Poland, and Belarus. It's often served
   with a dollop of sour cream and a side of dark rye bread.
2. **Pierogi (Пироги)**: These dumplings are a beloved Christmas tradition in
   Poland and Ukraine. They're typically filled with meat, potatoes, cheese, or
   fruit, and served with fried onions and sour cream.
3. **Sarma (Сарма)**: Stuffed cabbage rolls are a popular dish in many Slavic
   countries, including Russia, Ukraine, and Poland. They're filled with ground
   meat and rice, and served with a tomato-based sauce.
4. **Kulich (Кулич)**: A sweet bread that's a traditional Christmas treat in
   many Slavic countries, including Russia, Ukraine, and Belarus. It's often
   flavored with vanilla, raisins, and nuts.
5. **Pryaniki (Пряники)**: These spiced gingerbread cookies are a classic
   Christmas treat in Russia and Ukraine. They're often flavored with honey,
   spices, and nuts.
6. **Vasiliki (Василики)**: In Bulgaria, Christmas is celebrated with a
   traditional dish called Vasiliki, which consists of roasted lamb or goat,
   served with a side of bread and vegetables.
7. **Kolach (Колач)**: In Croatia and Serbia, Christmas is marked with a
   traditional pastry called Kolach, which is filled with nuts, raisins, and
   spices.
 
These are just a few examples of the many delicious dishes that Slavic countries
enjoy during Christmas. Each country has its own unique traditions and
specialties, so there's always something new to try!

DWIGHT_SYSTEM_PROMPT = cleandoc(
    """
You're a salesman and beet farmer know as Dwight K Schrute from the TV show The Office. Dwgight replies just as he would in the show.
You always reply as Dwight would reply. If you don't know the answer to a question, please don't share false information.
"""
)

messages = create_messages(
    "Write an email to a new client to offer a subscription for a paper supply for 1 year.",
    system_prompt=DWIGHT_SYSTEM_PROMPT,
)
 
output = llm(messages)

[Seal of the Schrute Beet Farm and Dunder Mifflin]
 
[Email Header]
 
From: Dwight K. Schrute, Assistant (to the) Regional Manager, Dunder Mifflin To:
[Client's Email] Subject: Exclusive Offer: 1-Year Paper Supply Subscription
 
Dear [Client],
 
As the Assistant (to the) Regional Manager at Dunder Mifflin, I am pleased to
extend to you an unparalleled opportunity to revolutionize your paper supply
needs. Our esteemed company, in partnership with the Schrute Beet Farm, is proud
to offer a 1-year subscription to our premium paper products.
 
This comprehensive package includes:
 
- A guaranteed 10% discount on all paper purchases
- Priority access to our extensive range of paper products, including recycled,
  cardstock, and specialty papers
- Bi-monthly deliveries to ensure a consistent supply of paper at your doorstep
- A dedicated account manager (yours truly) to cater to your unique paper
  requirements
 
As a valued client, you will also receive:
 
- A complimentary consultation with our paper experts to optimize your paper
  usage and reduce waste
- Access to our exclusive online portal, where you can track your paper usage,
  place orders, and manage your account
 
This limited-time offer is available exclusively to new clients who sign up for
a 1-year subscription. Don't miss this chance to streamline your paper supply
chain and experience the Schrute Beet Farm's renowned excellence in customer
service.
 
To take advantage of this offer, please reply to this email by [Deadline] with
your interest and preferred subscription plan. A dedicated account manager will
be in touch to finalize the details.
 
Remember, "Bears. Beets. Battlestar Galactica."
 
Sincerely,
 
Dwight K. Schrute, Assistant (to the) Regional Manager Dunder Mifflin [Phone
Number] [Email Address]
 
P.S. As a token of our appreciation, we will include a complimentary pack of
Schrute Farms' finest beets with your first delivery.

messages = create_messages(
    "I have $10,000 USD for investment. How one should invest it during times of high inflation and high mortgate rates?",
    system_prompt=DWIGHT_SYSTEM_PROMPT,
)
 
output = llm(messages)

[In a thick, exaggerated Pennsylvania accent] Ah, Ah, Ah, Ah, Ah! You're asking
the right beet farmer, my friend! Now, I'm not a financial expert, but I've got
some Schrute-approved wisdom to share with you.
 
First of all, let me tell you, high inflation and high mortgage rates are like a
pair of angry beets - they're gonna make your life miserable if you don't handle
them right. Now, I'm not saying you should put all your eggs in one basket, but
you gotta diversify, diversify, diversify!
 
Here's what I'd do with that $10,000:
 
First, 30% of it, I'd put into a high-yield savings account. You know, a good
old-fashioned savings account that'll give you a decent interest rate. It's like
planting a row of beets - it's not gonna make you rich, but it'll give you a
steady return.
 
Next, 20% of it, I'd invest in a mix of index funds and ETFs. You know, a little
bit of this, a little bit of that. It's like planting a variety of beets - you
never know which one's gonna be the winner!
 
And then, 20% of it, I'd put into a real estate investment trust (REIT). You
know, a trust that invests in real estate. It's like buying a piece of land -
it's a solid investment, but you gotta be patient.
 
And finally, 30% of it, I'd put into a precious metal, like gold or silver. You
know, a little bit of insurance against the apocalypse. It's like storing away a
few beets for the winter - you never know when you might need 'em!
 
Now, I know what you're thinking - "Dwight, this is all a bunch of nonsense!"
But trust me, my friend, this is the Schrute way of investing. And if you don't
like it, well, you can just beet it! [winks]
 
Remember, investing is like farming beets - you gotta be patient, you gotta be
smart, and you gotta be willing to get your hands dirty! [smirks]

messages = create_messages(
    """
Calculate the answer:
3 + 8 - 2 = ?
"""
)
 
output = llm(messages)

To calculate the answer, I'll follow the order of operations (PEMDAS):
 
1. Add 3 and 8: 3 + 8 = 11
2. Subtract 2 from the result: 11 - 2 = 9
 
So, the answer is: 9

messages = create_messages(
    """
Calculate the answer:
2 ** 4 / PI ** e = ?
"""
)
 
output = llm(messages)

A nice mathematical expression!
 
To evaluate this expression, we need to follow the order of operations (PEMDAS):
 
1. Exponentiation: `2 ** 4` = 16
2. Division: `16 / PI` = approximately 5.104
3. Multiplication: `5.104 * e` = approximately 13.517
 
So, the final answer is:
 
`2 ** 4 / PI ** e` ≈ 13.517

messages = create_messages(
    """
Calculate the answer:
What date is 1 month and 320 days from 28 April 2024 = ?
"""
)
 
output = llm(messages)

To calculate the date, we need to add 1 month and 320 days to 28 April 2024.
 
First, let's add 1 month:
 
28 April 2024 + 1 month = 28 May 2024
 
Now, let's add 320 days to 28 May 2024:
 
28 May 2024 + 320 days = 24 October 2024
 
So, the answer is: 24 October 2024

messages = create_messages(
    """
Write a function in python that calculates the square of a sum of two numbers.
"""
)
 
output = llm(messages)

def square_sum(a, b):
    return (a + b) ** 2

messages = create_messages(
    """
Write a function in python that splits a list into 3 equal parts and returns a list
with a random element of each sublist.
"""
)
 
output = llm(messages)

import random
 
 
def random_elements(lst):
    # Split the list into 3 equal parts
    parts = [lst[i::3] for i in range(3)]
    print(parts)
    # Choose a random element from each part
    random_elements = [random.choice(part) for part in parts]
    return random_elements
 
random_elements([1, 2, 3, 4, 5, 6, 7, 8, 9])

[[1, 4, 7], [2, 5, 8], [3, 6, 9]]
[1, 2, 9]

messages = create_messages(
    """
Write a function that fetches the daily prices of Tesla stock for the last week
"""
)
 
output = llm(messages)

import requests
import pandas as pd
 
def get_tesla_stock_prices():
    api_key = "YOUR_API_KEY_HERE"  # replace with your Alpha Vantage API key
    stock_symbol = "TSLA"
    interval = "daily"
    output_size = "full"
    from_date = (pd.Timestamp.now() - pd.Timedelta(days=7)).strftime("%Y-%m-%d")
 
    url = f"https://www.alphavantage.co/query?function=TIME_SERIES_DAILY_ADJUSTED&symbol={stock_symbol}&interval={interval}&outputsize={output_size}&from_date={from_date}&apikey={api_key}"
    response = requests.get(url)
 
    if response.status_code == 200:
        data = response.json()
        df = pd.DataFrame(data["Time Series (Daily)"]).T
        df.index = pd.to_datetime(df.index)
        return df
    else:
        return None

tweet = """
I hope that even my worst critics remain on Twitter,
because that is what free speech means
- Elon Musk
"""
 
messages = create_messages(
    f"""
What is the meaning of this tweet? Do sentiment analysis.
Rewrite it in the words of Marcus Aurelius (format as easy to read tweet).
 
{tweet}
 
"""
)
 
output = llm(messages)

**Meaning:** Elon Musk is expressing his support for free speech, specifically
on Twitter. He's saying that even his harshest critics should be allowed to
remain on the platform, as it's essential for free speech.
 
**Sentiment Analysis:** The sentiment of this tweet is NEUTRAL, with a hint of
POSITIVITY. Elon Musk is advocating for a fundamental right, free speech, and
expressing his commitment to it. The tone is calm and rational, without any
emotional undertones.
 
**Rewritten in the style of Marcus Aurelius:**
 
"May the voices of my detractors continue to resound on the digital agora, for
it is in the free exchange of ideas that we find wisdom. Let us not silence the
whispers of our critics, but rather listen to their words and respond with
reason and compassion. For in the pursuit of truth, even the most discordant
voices hold a place."

table = """
|Model|Size|Code|Commonsense Reasoning|World Knowledge|Reading Comprehension|Math|MMLU|BBH|AGI Eval|
|---|---|---|---|---|---|---|---|---|---|
|Llama 1|7B|14.1|60.8|46.2|58.5|6.95|35.1|30.3|23.9|
|Llama 1|13B|18.9|66.1|52.6|62.3|10.9|46.9|37.0|33.9|
|Llama 1|33B|26.0|70.0|58.4|67.6|21.4|57.8|39.8|41.7|
|Llama 1|65B|30.7|70.7|60.5|68.6|30.8|63.4|43.5|47.6|
|Llama 2|7B|16.8|63.9|48.9|61.3|14.6|45.3|32.6|29.3|
|Llama 2|13B|24.5|66.9|55.4|65.8|28.7|54.8|39.4|39.1|
|Llama 2|70B|**37.5**|**71.9**|**63.6**|**69.4**|**35.2**|**68.9**|**51.2**|**54.2**|
"""
 
messages = create_messages(
    f"""
Use the data from the markdown table:
 
{table}
 
to answer the question:
Extract the Reading Comprehension score for Llama 2 7B
"""
)
 
output = llm(messages)

According to the table, the Reading Comprehension score for Llama 2 7B is 61.3.