• 0 Posts
  • 31 Comments
Joined 3 years ago
cake
Cake day: June 26th, 2023

help-circle


  • And then you have a trained model that requires vast amounts of energy per request, right? It doesn’t stop at training.

    You need obscene amounts GPU power to run the ‘better’ models within reasonable response times.

    In comparison, I could game on my modest rig just fine, but I can’t run a 22B model locally in any useful capacity while programming.

    Sure, you could argue gaming is a waste of energy, but that doesn’t mean we can’t argue that it shouldn’t have to cost boiling a shitload of eggs to ask AI how long a single one should. Or each time I start typing a line of code for that matter.

















  • Well, it’s meant as an introductory paragraph. I think such a general paragraph should not go to those lengths since the vast majority won’t be facing that issue. Most large instances that you would recommend for first timers are federated well enough that at least the civilised part of Lemmy is very accessible. I think that with:

    • sensible defaults/suggestions
    • easy to understand intro
    • a help/link to a detailed article

    you cover enough for users who can’t be bothered, who want to be informed, and those who want to understand what’s going on behind the scenes.