Sampreeth Avvari
Resume
Read the story

Architecture

Labels election tweets true, false or opinion.On one V100 GPU.

A 4 bit Llama 3 8B, fine tuned on LIAR claims, labels 2020 election tweets True, False or Personal with a one token answer. RoBERTa adds a mood to each tweet, and the counts become state maps, swing state charts and a topic table.

1 token

generated per tweet, so each verdict is one decoding step

315,020 tweets

found under both hashtags after cleaning, removed from both before labeling

4 days

from the first commit to the final report, May 8 to 12, 2024

76 %

accuracy on LIAR test claims, true or false, as the report states

New York --:-- simulated

A tweet
Election Fake News A 4 bit Llama 3 8B, fine tuned on LIAR claims, labels 2020 election tweets True, False or Personal with a one token answer. RoBERTa adds a mood to each tweet, and the counts become state maps, swing state charts and a topic table. LIAR benchmark 10,240 training claims QLoRA fine tune Llama 3 8B, 4 bit Hugging Face 336 MB adapter Election tweets Oct 15 to Nov 8, 2020 Clean and dedupe drop shared texts Llama classifier 12 shot, one token RoBERTa sentiment 3 moods, zero shot Power BI maps states and swing states LDA topics per label and mood Final report 9 topic themes claims adapter 4 bit + adapter tweets clean text with a label plus a mood groups topic table figures NYU Greene HPC, one V100

Hover a part to trace what it touches. Click it for its notes.

  • Built by me
  • Managed by Google
  • Outside system
  • Main data path
  • On demand
  • Credentials cross

When it breaks

What each failure does, and what it does not do.

  1. Answers off the label list

    A one token answer is not always a label. On one run 2,945 answers came back as four hash marks.

    fake_news_detection_inference.ipynb

  2. Same tweet, both hashtags

    315,020 cleaned tweets appeared in both files. Kept, they would count once for each candidate.

    fake_news_detection_inference.ipynb

  3. No answer key for tweets

    Accuracy can only be measured on LIAR. The tweets carry no labels, and the report says so.

    Final Report.pdf

What I tuned

Small choices that keep it fast and cheap.

  1. 4 bit base, small adapter

    QLoRA freezes the 4 bit base and trains 83.9 million adapter parameters, so the run fits on one 32 GB V100.

    fake_news_detection-finetuning.ipynb

  2. Six labels down to two

    Training ended with 5,545 false and 4,695 true claims after half true was split at random.

    fake_news_detection-finetuning.ipynb

  3. Learning rate cut to 8e-6

    Early runs used 2e-4. The final run used 8e-6 over 30 epochs and 38,400 steps.

    fake_news_detection-finetuning.ipynb

Guards

Rules the code enforces, not rules people remember.

  1. One prompt shape

    Claims and tweets go through the same instruction, input and response template the adapter was trained on.

    fake_news_detection_inference.ipynb

  2. English only for mood

    Language detection drops non English tweets before RoBERTa scores them.

    Sentiment Analysis/Sentiment Analysis.ipynb

  3. Mentions and links masked

    Handles become @user and links become http, the form the RoBERTa model expects.

    Sentiment Analysis/Sentiment Analysis.ipynb