I’ve read a lot of hardware discussions here, but never a mention of cards like AMD 102D1631200 Radeon Instinct MI60 32GB HBM2 . Their VRAM and bandwidth are certainly fine, the price more than fine (used only), is there a devil in the details?

  • hexaga@alien.top
    BEnglish
    1·
    1 year ago

    They work fine but have a couple pitfalls:

    • rocm support across software is spotty, but does work when available (mostly)
    • rocm is less well supported os wise (have to use specific, blessed distros / kernels)
    • most llm software is optimized for nvidia hardware, perf is worse than it could be w/ bespoke cuda kernels