Post by @brucethemoose

In reply to 3 earlier posts

@alphacyberranger@sh.itjust.works on sh.itjust.works Open parent

917

207

@brucethemoose@lemmy.world on lemmy.world Open parent

Aside: WTF are they using SSDs for? LLM inference in the cloud is basically only done in VRAM. Rarely stale K/V cache is cached in RAM, but new attention architectures should minimize that. Large scale training, contrary to popular belief, is a pretty rare event most data centers and businesses are incapable of. …So what do they do with so much flash storage!? Is it literally just FOMO server buying?

Open parent Original URL

@T156@lemmy.world on lemmy.world Open parent

Storage. There aren’t enough hard drives, so datacentres are also buying up SSDs.

Open parent Original URL

-2

brucethemoose in !technology

@brucethemoose@lemmy.world · Dec 15

since it’s needed to store training data. Again, I don’t buy this. The training data isn’t actually that big, nor is training done on such a huge scale so frequently.

View on lemmy.world

Comments (1)

Showing 0 of 1 cached locally.

Syncing comments from the remote thread. 1 more reply is still loading.

Loading comments...

About Community

Technology

!technology@lemmy.world

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related news or articles.
Be excellent to each other!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
Check for duplicates before posting, duplicates may be removed
Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

83902

Members

18816

Posts

Created: June 11, 2023

View All Posts