Today I saw a fun open-source project called minigp, and I’m gonna give it a try.
First, I grabbed the code from the project page. It’s pretty straightforward, just a few files. I then opened my code editor and took a look at the code. It’s all Python, which is great ’cause that’s what I’m good at.
I started to play around with it by running their example scripts. They have some basic ones that show you how it works. I ran `*`, it seemed no problem. And the script started training a model, and I could see the loss going down. Cool!
I also tried to run `*`, it worked as well. It generated some text based on the training text data that I just put in. It was kinda gibberish, but hey, it’s a start, right? It generated some text that kinda looked like Shakespeare. Funny stuff!
Then, I wanted to see if I could tweak things a bit. I went into the `*` file and started messing with the parameters. Changed the number of layers, the learning rate, things like that. Just experimenting, you know?
- I changed the `n_layer` from 6 to 8.
- I changed the `n_head` from 6 to 8.
- I changed the `n_embd` from 384 to 512.
After I made those changes, I ran the training script again. It took a little longer to train, ’cause I increased the model size, but it seemed to be learning better. The loss was lower than before. Progress!
Next, I tried to sample the text generation with the modified setting. I used the same command and gave it a different prompt. I was curious to see how that would change the output. The results were different, for sure. Still kinda nonsensical, but it felt like it was generating text that was a bit more complex.
To be honest, I spent hours just playing around with different settings and different prompts. It’s pretty addictive, just seeing what this little model can do. You can try to change these parameters to see how they affect the training and generation.
In summary, I tried out this project called minigp, played around with it, messed with some settings, and saw how it changes the output. It’s really cool to see how these models work under the hood. I gotta say, I learned a lot today just from messing around with this thing. I hope you can also get your hands on it and try it out yourselves!
What I Did Today
- Grabbed the code and ran the example scripts.
- Tried to modify the config parameters and observed the changes.
- Sampled different text generations and compared the outputs.
Hope you guys enjoy it!
