Installing AMDGPU drivers with OpenCL support on Linux

[biodoc]

Here's what I did to get my new Radeon VII going on Linux.

1) Clean install of Ubuntu 18.04.4 LTS. I tried to get the open source AMDGPU drivers on my existing installation on linux 19.3 (nvidia drivers) but I haven't succeeded yet. I'm making progress I think but time will tell if I'm successful.

1a) Install boinc and system updates
sudo apt install boinc-client boinc-manager
sudo usermod -a -G boinc $(whoami) <---adds user (you) to the boinc group
sudo apt update
sudo apt upgrade
sudo reboot

2) The clean install of Ubuntu 18.04.4 LTS automatically installs the AMGGPU opensource drivers but that alone will not support OpenCL apps. This is confirmed by installing boinc and sure enough, "no usable gpus detected" or something similar was reported in the event log. To install OpenCL drivers, download the latest AMDGPU-Pro driver package from https://www.amd.com/en/support. The driver file I downloaded was amdgpu-pro-19.50-967956-ubuntu-18.04.tar.xz at this url. https://www.amd.com/en/support/graphics ... adeon-vii.

3) Extract the archive.
$ cd ~/Downloads
$ tar -Jxvf amdgpu-pro-19.50-967956-ubuntu-18.04.tar.xz


4) Change directory and run the installer to install just the openCL support files.

$ cd ~/Downloads/amdgpu-pro-19.50-967956-ubuntu-18.04
Run:
sudo ./amdgpu-pro-install --opencl=legacy,pal --headless --no-dkms

5) reboot or restart the boinc-client service: sudo service boinc-client restart
6) check the boinc event log to see if your GPU was detected.


The only option app I could find to control voltage, clock speed, etc that supported the Radeon VII was amdgpu-utils whcih can be found here:

Ricks-Lab/amdgpu-utils

A set of utilities for monitoring and customizing GPU performance - Ricks-Lab/amdgpu-utils

github.com

This was really tedious to install. Make sure you read the user guide in detail.

 

[biodoc]

I was not able to get F@H running on my radeon VII on the linux rig that works for all the boinc GPU projects using the installation method above. My guess is that user 'fahclient' does not have permission to load the OpenCl drivers, because I was able to tun FAH when I started the client manually in my home directory. If you'd like to try this here's what I did.

EDIT: The fix for this issue is in this thread at the folding forum.

sudo systemctl disable FAHClient <----disables the FAHClient service that starts automatically during boot.
sudo reboot
In your home directory make folder FAH: mkdir FAH
cd FAH
wget http://fah-web.stanford.edu/file-releases/public/GPUs.txt
cp /etc/fahclient/config.xml ./
FAHClient <---- this starts the client

I know this isn't ideal but it works for me.

 

[Assimilator1]

Sorry, not related, but a Radeon VII has awesome DP , the HD 7970 is finally beaten by it! (consumer cards). Anyway, my question is, presumably you ran MW on it at some point, what times were you getting?

 

[StefanR5R]

I was not able to get F@H running on my radeon VII on the linux rig that works for all the boinc GPU projects using the installation method above. My guess is that user 'fahclient' does not have permission to load the OpenCl drivers, because I was able to tun FAH when I started the client manually in my home directory.

Does id boinc show more extensive group membership than id fahclient?

 

[biodoc]

Does id boinc show more extensive group membership than id fahclient?

That's once of the things I checked. I added fahclient to the video and render groups but that didn't help. boinc is a member of the boinc group but fahclient is a member of nogroup.

mark@linux-x32:/etc/fahclient$ id boinc
uid=122(boinc) gid=127(boinc) groups=127(boinc),44(video),132(render),134(vboxusers)
mark@linux-x32:/etc/fahclient$ id fahclient
uid=127(fahclient) gid=65534(nogroup) groups=65534(nogroup),44(video),132(render)

mark@linux-x32:/etc/fahclient$ groups mark
mark : mark adm cdrom sudo dip video plugdev lpadmin sambashare boinc

 

[bwoochowski]

Just wanted to report that fahclient is working on rocm 3.3. The rocm suite comes with the rocm-smi utility for easy monitoring and management.

ROCm Installation v4.3 — ROCm 4.5.0 documentation

Forgot to add you'll need to install these additional packages from the Ubuntu repos:
ocl-icd-opencl-dev
clinfo (even though rocm comes with it's own clinfo which will be in your path if following the linked instructions)

Don't forget to add fahclient to the video group.

 

[biodoc]

Sorry, not related, but a Radeon VII has awesome DP , the HD 7970 is finally beaten by it! (consumer cards). Anyway, my question is, presumably you ran MW on it at some point, what times were you getting?

Sorry I missed this question. I'll have to give a fuzzy answer. I've been controlling high GPU temps by setting the fan speed at around 55% (tolerable noise for me) and setting power limits. I finally settled on a power cap of 150 watts for 24/7 use. I did run Milky Way some time ago but I don't remember what power cap I was using at the time. When I did test it, I was seeing 40 sec run times crunching 4 tasks simultaneously. I suspect that number will be higher at a 150 watt power cap.

Just wanted to report that fahclient is working on rocm 3.3

That's good news. I'll have to give it a test. Do you know if rocm supports Navi gpus yet?

 

[bwoochowski]

From what I've read Navi support is very limited, probably best to avoid on that generation when using the rocm stack.

 

[JustMe21]

I was not able to get F@H running on my radeon VII on the linux rig that works for all the boinc GPU projects using the installation method above. My guess is that user 'fahclient' does not have permission to load the OpenCl drivers, because I was able to tun FAH when I started the client manually in my home directory. If you'd like to try this here's what I did.

sudo systemctl disable FAHClient <----disables the FAHClient service that starts automatically during boot.
sudo reboot
In your home directory make folder FAH: mkdir FAH
cd FAH
wget http://fah-web.stanford.edu/file-releases/public/GPUs.txt
cp /etc/fahclient/config.xml ./
FAHClient <---- this starts the client

I know this isn't ideal but it works for me.

You can do this instead so you don't have to disable the FAHClient.

wget https://fah-web.stanford.edu/file-releases/public/GPUs.txt

sudo mv GPUs.txt /var/lib/fahclient/

sudo service FAHClient restart

 

[StefanR5R]

You can do this instead so you don't have to disable the FAHClient.
[...]

The trouble was not with GPUs.txt, but with the loading of the OpenCL driver on that system (if FAHClient was started by systemd, but not if it was started by the main user).

@biodoc, if you find out with which step the driver can be loaded, you may be able to add this step to the service config file (located in /lib/systemd/system, or etc/init.d, or...?) if you prefer to start/stop the client as a service.

 

[biodoc]

The trouble was not with GPUs.txt, but with the loading of the OpenCL driver on that system (if FAHClient was started by systemd, but not if it was started by the main user).

@biodoc, if you find out with which step the driver can be loaded, you may be able to add this step to the service config file (located in /lib/systemd/system, or etc/init.d, or...?) if you prefer to start/stop the client as a service.

Here's the fix in this thread. He created a systemd unit to start FAHClient instead of the init.d script. This works by using the systemd unit's "User" option to change users instead of having FAHClient do it. There are some amazing linux gurus in the folding forum. Good thing we have @StefanR5R too to point me in the right direction.

 

[TennesseeTony]

Welllllll, crap. Okay got myself in a bind. How do I start the GUI, reboot ends up at a console/terminal style text asking for login and password. I managed to login, and even managed to reinstall the driver. Rebooted again and still no GUI.

 

[StefanR5R]

I guess X11 failed to start.
On Mint 18.3 Cinnamon, more info can be had with
service lightdm status
less /var/log/Xorg.0.log

 

[biodoc]

Welllllll, crap. Okay got myself in a bind. How do I start the GUI, reboot ends up at a console/terminal style text asking for login and password. I managed to login, and even managed to reinstall the driver. Rebooted again and still no GUI.

More details please. Which GPU? Fresh install of Mint 19,3? Did it boot to console after installation or after installation of the AMDGPU-Pro opencl drivers?

 

[TennesseeTony]

Radeon VII, fresh Mint 19.3 Cinammon (changed to Mate later), downloaded AMD Drivers and installed.
Couldn't figure out how to install the opencl part, decided I had best ask doc.
Remembered doc already made a thread about this.
Followed doc's instructions, but it said can't install, you already have an installation. Perhaps uninstall and run again.
Hmm. Simple enough. Uninstalled, rebooted, and....stuck at the terminal login.
Did what Stephan said, got all sorts of info I didn't understand.
Did what I always do, got impatient/aggrevated and did a fresh install, this time with 19.3 MATE.
Starting from scratch now, even though I know you said to try Ubuntu. The drivers are newer, so, fingers crossed.

A short while later:
Good to go for BOINC.
Einstein is offline?!?!?!?
Tested with a MW wu, for @Assimilator1, got 16 seconds for one. Previously on Windows 3 at a time took 32 seconds, Mark(assimilator1).

edit: More MW times:
Single task 16,15,15,16 seconds, so...15.5?
Triple tasks: 28,28,28,31,31,31,32,32,32,34...uhm 31-ish seconds? Less than 11 seconds per task in other words, down from 15.5.
Quad tasks: 40,46,42,38,44,38,38,34,38,38,40,34... 39.2 seconds each, so less than 10 seconds per task. I will stop at quad tasks.


Now I need to use this post as a scratch pad, pardon the mess:

<app_config>
<app>
<name>milkyway</name>
<gpu_versions>
<gpu_usage>0.25</gpu_usage>
<cpu_usage>0.05</cpu_usage>
</gpu_versions>
</app>
</app_config>

 

[Assimilator1]

Tested with a MW wu, for @Assimilator1, got 16 seconds for one. Previously on Windows 3 at a time took 32 seconds, Mark(assimilator1).

That's quick!
What credit WU was it?

 

[TennesseeTony]

For Stephan:
The 2nd command failed. I tried 0 and O (number and letter) in Xorg.0.log

Here is lightdm:

 

[biodoc]

Quad tasks: 40,46,42,38,44,38,38,34,38,38,40,34... 39.2 seconds each, so less than 10 seconds per task. I will stop at quad tasks.

I run quad tasks on MW too. Great that you got your shiny new VII working on linux.

 

[TennesseeTony]

It appears one thread isn't enough for 3 or 4 tasks, had to give it two threads, and the times returned to the above mentioned times, well, actually a little bit slower. Now that the CPU is loaded, the CPU clock has presumably dropped. First card I have seen that the CPU time makes a difference in MW! lol edit: And I blew through hundreds of tasks and ran out. Milkyway's update cycle is terrible.

So what about Einstein...is it down? This is day 3 that I can't reach it.

 

[StefanR5R]

I just tried an [Update] on one of my computers which is already attached to Einstein. The scheduler request succeeded.

I've got this project URL: http://einstein.phys.uwm.edu/

Forum link: https://einsteinathome.org/community

Kiska's site: https://munin.kiska.pw/munin/einstein-week.html

If Einstein@home were down, you can bet that the Pentathlon shoutbox would have been all abuzz about it already.

 

[TennesseeTony]

Interesting. Perhaps I once managed to succeed at some firewall blocking. Everything down the line past my 2nd router/wireless booster gets this:

 

[biodoc]

Maybe there's a clue in your router log.

 

[TennesseeTony]

Ha. There be the problem. I will stop hijacking this thread now. Except to say Einstein on a 2080Ti=6m57s ; On the Radeon VII=3m35s. WOW. Even better with dual tasks, just under 6m for a pair.