And I use an MI50 for AI tasks, including testing LLMs. Performance is 34tokens/s on 13B models. 33B models the speed drops to around 8 tokens/s. the MI50 only has 16GB of VRAM.
ROCm compatibility has improved a lot this year, openCL support is very good. Even openMP’s offload support is very good and I’m using it in some personal projects, the use of HBM2 memory gives a good boost in certain computing-intensive tasks.
However, this does not apply to Windows, it is still very unstable and MI50/60 are not officially supported. The second option is to use DirectML, but all the solutions seem to be a house of cards that anything causes the system to stop working.
An important observation is the bios used on these boards. The ones I have have two bios installed, one of them is the modified mining version that causes abnormal heating, changing the bios switch, everything returns to normal.
And I use an MI50 for AI tasks, including testing LLMs. Performance is 34tokens/s on 13B models. 33B models the speed drops to around 8 tokens/s. the MI50 only has 16GB of VRAM.
ROCm compatibility has improved a lot this year, openCL support is very good. Even openMP’s offload support is very good and I’m using it in some personal projects, the use of HBM2 memory gives a good boost in certain computing-intensive tasks.
However, this does not apply to Windows, it is still very unstable and MI50/60 are not officially supported. The second option is to use DirectML, but all the solutions seem to be a house of cards that anything causes the system to stop working.
An important observation is the bios used on these boards. The ones I have have two bios installed, one of them is the modified mining version that causes abnormal heating, changing the bios switch, everything returns to normal.