BOINC: Windows to Linux

[Howdy]

Yup, I got the TV ban for "commercial usage" too.
Now after I say Any Desk is good.......Using a VPN and Any Desk I can connect remotely but not control and lose connection. (Using a "closer" area has "fixed" this) If I disable the VPN all is good....I believe it's because scammers are now using it to screw people.

 

[crashtech]

So the reason I haven't ever moved all my machines to Linux, and still maybe won't, is because literally nothing works, except what is pre-installed. Everything is a Google search about why doesn't y do x when it seems to install normally? It's been like this since the nineties when I tried it for the first time, and though things are better today, it's still a giant pain in the ass.

 

[VirtualLarry]

So the reason I haven't ever moved all my machines to Linux, and still maybe won't, is because literally nothing works, except what is pre-installed. Everything is a Google search about why doesn't y do x when it seems to install normally? It's been like this since the nineties when I tried it for the first time, and though things are better today, it's still a giant pain in the ass.

That's basically been my experience over the years as well. It's "getting better", constantly, but new waves of hardware, and compatibility issues always seem to abound, for hardware that "just works" in Windows.

Those whom have tried to implement GPU mining / DC on AMD cards in Linux can kind of attest, I think, to what we're both saying.

Nvidia - so I've heard, haven't tried lately, is much easier to get going. In fact, I might try it for the f@H race. Maybe.

 

[crashtech]

@VirtualLarry , I did have one Linux box going for a year or so, but that was about the time TeamViewer started messing with me and it was the only cross platform remote desktop software that I knew how to make work. At this moment, it still is. It seems like almost everyone on the TeAm is gung-ho on Linux, so I'm going to try yet again, but it gives me a headache.

 

[crashtech]

Here's an example of the inscrutability of Linux: I've installed BOINC several times on Linux without incident, but this time, on Mint 19.2 "Tina," BOINC Manager works fine, IF I go to "Select Computer" and type in a loopback address. Otherwise, it just sits there. I'm sure there is an easy cure, but why should there have to be one?

 

[crashtech]

Well, I did finally get Chrome Remote Desktop to handle incoming connections to my test Mint box, sort of. It starts its own session, separate from the main login session, which has hardware acceleration disabled with no way to install drivers successfully. Once this installation occurred, I could log in remotely but not locally, the local login screen just kicks me out. That might be fine, but the remote session seems crippled. Trying to switch from wired to wifi (needed for this boxes location) proved too much for my skills, buried in an avalanche of authentications followed by a lack of connectivity each time it was attempted in slightly different ways. I'm left with the impression that Linux is still the best OS around, if you are highly proficient with the CLI. Which, I sort of am, at least compared to anyone I know irl, but that's woefully inadequate!

Weighing my options while 20 cores are sitting here, dark.

 

[biodoc]

@crashtech, I don't use remote desktop software to connect to other linux boxes on my home network but I can walk you through what I do to manage my "remote" linux computers. @StefanR5R , please check and make your own recommendations please.

I'm assuming your primary computer runs windows and your remote computer runs a recent version of Mint linux. The first goal is to be able to connect to the boinc client on the linux machine with boinc manager running on the windows machine. The second goal is to set up an Openssh server on the linux machine to be able to log in to that machine from your windows machine using an openssh client like puTTY. This will allow you to open up a secure shell from windows to linux to execute commands on the linux machine like restarting the boinc-client.

Ideally your network should be set up to use fixed IP addresses on your DC computers. If you use a DHCP server to assign IP addresses, like me, it's not a deal breaker but keep in mind if you have trouble connecting to a remote machine, it's likely the IP address has changed. My memory is not as good as it used to be so you may want to keep a text file on your windows machine that lists IP addresses of your remote machines and another text file to store useful linux commands along with a brief description of what they do.

Things you need to do on the linux machine:

Spoiler

sudo apt install openssh-server -->installs secure shell server software which allows connection from a remote machine.

Start using and learning nano as a terminal text editor. It's necessary to edit files on the remote linux machine via an SSH session from your windows machine.

sudo nano /var/lib/boinc-client/cc_config.xml

Add <allow_remote_gui_rpc>1</allow_remote_gui_rpc> to the options section.

CTRL-O to save file and CTRL-X to exit nano.

sudo nano /var/lib/boinc-client/gui_rpc_auth.cfg

add your remote connection password to the file.

sudo nano /etc/boinc-client/remote_hosts.cfg

add the IP address of your main windows machine

sudo service boinc-client restart

sudo apt install boinctui

Installs boinctui which is an alternative boinc manager that you may be able to use over an SSH connection. Linux to linux works but I haven't tested windows to linux.

Find the IP address of the remote linux machine:

ifconfig

On the windows machine:

In boinc manager, under File, choose 'select computer'. Type in the IP address of the remote linux machine and enter the password. Now you can manage the boinc-client on the remote machine.

Find and install puTTY (see link above). Open up puTTY and follow the instructions at the link above to connect to the linux machine (you'll need the IP address). Once you have an SSH connection, you can manage the remote computer via bash shell commands. Try running boinctui to see if that works. Other useful linux apps to run:

top --> monitors RAM/CPU usage
htop --> same as top but has a peudo graphical interface that is cool and useful.
nano --> a text editor for editing files on the remote machine
sudo apt update --> checks for new versions of apps/libs/kernels, etc.
sudo apt upgrade --> install the new versions. (If a new linux kernel is installed, you'll need to reboot)
sudo reboot --> reboots the remote computer
sudo shutdown --> powers down the remote computer
lscpu ---> check cpu frequency
nvidia-smi ---> check status of nvidia cards

 

[crashtech]

Thanks, biodoc, for taking the time to put that together for me! Maybe I will try that out tonight when I get home. Since I have machines at two locations, it doesn't seem like I will be accessing ones on a different LAN with the above method, but conceivably I could use conventional remote desktop software to connect to a Windows machine on the other LAN, then use it to access the Linux boxes there.

 

[biodoc]

You can use SSH to connect to a computer anywhere and the same is true for boinc remote access. I don't have any experience setting up communication between 2 different LANs though.

 

[crashtech]

Well, I gather you have to use port forwarding and possibly have to interact with the ISP as well, something that I'm reluctant to do. I'm pretty reluctant to do any of it, really, but I'm tired of leaving so much performance on the table.

 

[StefanR5R]

@StefanR5R, please check and make your own recommendations please.

I can't really help because my setup is so much simpler than what @crashtech needs.

Spoiler: what I use and am familiar with

• I access all m computers only in a single LAN segment. Accessing them remotely would be very difficult to get working, due to double NAT (local cable modem/ router, and at the ISP out of my control).
• I configured fixed IP addresses on all computers. Fixed adresses could be handed out by a DHCP server too (e.g. the one on my cable modem/ router), but I'm set in my ways.
• In the rare case when I have to go into the BIOS, but also if I want to manage anything via GUI, I can hook up a monitor via a small KVM switch.
• I control boinc mostly remotely through boincmgr, boinccmd, boinctasks. The latter I use almost exclusively from a Windows laptop, since it is too sluggish to my taste when running with Wine on Linux.
• For boinc control that cannot be done this way, e.g. setup of app_info.xml and such, I use scp and ssh. (scp because I have been too lazy so far to set up a networking filesystem, such as NFS, on my boinc appliances. I used NFS extensively in the past though for other purposes, and it was easy enough for me to set up.)
• Hence, one of the very first steps which I go through when I set up a new boinc appliance, which happens rarely, is to install and enable sshd. On Linux Mint, I simply perform this particular step through the GUI software manager.
• I have been using mostly Gentoo Linux in the past, and this is managed practically exclusively through the command line, and to a large extent with Gentoo-specific tools. For boinc, I have recently started to use other Linux OSs again (Linux Mint 18.3 only for NVidia-equipped computers, OpenSuse when I need VirtualBox on GPU-less computers because I am too lazy to deal with VBox on Gentoo). Since I am largely unfamiliar with these other Linux flavors, I manage them mostly through their native GUI tools.

I use Windows at my day job, but don't understand it deeply. And how could I: Unlike Linux, it is largely undocumented and wildly inconsistent.

I started using Unix and Linux, but also DOS/Windows, in the 1990s.

In the past, I used remote GUIs such as X11, RDP, VNC; but not anymore, as I have no need.

I generally don't use closed-source, anti-libre software at home. NVidia's driver for GPU computing is the big exception (and one rarely used remaining Windows laptop and an Android based cell phone; I have almost no personal data on either of them).

 

[crashtech]

I'm going to keep trying to wear my big boy trousers. I nuked and re-installed my borked Mint install this morning, and got boinc and boinctui working before having to leave for work.

Connecting two Windows boxes together with conventional desktop software to take care of access outside the LAN should not be too terrible, as I am guessing the local ssh sessions should be fairly lag-free. It seems by far the simplest solution.

Setting static IPs on my LAN would seem to make good sense, although long ago I broke connectivity by doing something wrong, and haven't tried again since. Some of my antics play havoc with the Internet access of others in the house, making such things dicey. Last time I tried anything was with a Pi-hole, which I got working on the wired portion of the local network but it broke wireless for some reason I was never given enough time to discover. Kinda makes me wonder if I could get a second IP from my ISP such that I could mess up as much as I wanted to and not affect the wife and kids...

 

[biodoc]

One possible solution is to set up dynamic DNS on both networks and go from there. I would worry about security though. I checked my router and it does support DDNS.

 

[crashtech]

I'm going to re-copy the tutorial here with some changes that I've needed to make as I go, consider it a work in progress!

Spoiler

Install video drivers:

sudo apt-get install nvidia-390 (or 430 for 20-series)

sudo apt install openssh-server -->installs secure shell server software which allows connection from a remote machine.

sudo apt install boinc

Start using and learning nano as a terminal text editor. It's necessary to edit files on the remote linux machine via an SSH session from your windows machine.

Since I am a GUI guy, I also will install gedit: sudo apt install gedit

sudo nano /var/lib/boinc-client/cc_config.xml

Add <allow_remote_gui_rpc>1</allow_remote_gui_rpc> to the options section.

Current cc_config in use, although it may be helpful to allow CPU benchmarks one time before this implementation:

<cc_config>
<options>
<allow_remote_gui_rpc>1</allow_remote_gui_rpc>
<use_all_gpus>1</use_all_gpus>
<allow_multiple_clients>1</allow_multiple_clients>
<skip_cpu_benchmarks>1</skip_cpu_benchmarks>
<ncpus>100</ncpus>
</options>
</cc_config>

CTRL-O to save file and CTRL-X to exit nano.

sudo nano /var/lib/boinc-client/gui_rpc_auth.cfg

add your remote connection password to the file.

sudo nano /etc/boinc-client/remote_hosts.cfg /var/lib/boinc-client/remote_hosts.cfg

add the IP address of your main windows machine

sudo service boinc-client restart

sudo apt install boinctui

Installs boinctui which is an alternative boinc manager that you may be able to use over an SSH connection. Linux to linux works but I haven't tested windows to linux.

add your remote connection password to boinctui under File>Configure Host List

Find the IP address of the remote linux machine:

ifconfig

Install OpenCL dev driver:

sudo apt install ocl-icd-opencl-dev

Install OpenGL Utility Toolkit:

sudo apt-get -y install python-gnome2 mesa-common-dev freeglut3-dev nvidia-settings gedit

Install fahclient and fahcontrol, then:

cd /var/lib/fahclient
sudo wget http://fah-web.stanford.edu/file-releases/public/GPUs.txt
sudo service FAHClient stop
sudo service FAHClient start

On the windows machine:

In boinc manager, under File, choose 'select computer'. Type in the IP address of the remote linux machine and enter the password. Now you can manage the boinc-client on the remote machine.

Find and install puTTY (see link above). Open up puTTY and follow the instructions at the link above to connect to the linux machine (you'll need the IP address). Once you have an SSH connection, you can manage the remote computer via bash shell commands. Try running boinctui to see if that works. Other useful linux apps to run:

top --> monitors RAM/CPU usage
htop --> same as top but has a peudo graphical interface that is cool and useful.
nano --> a text editor for editing files on the remote machine
sudo apt update --> checks for new versions of apps/libs/kernels, etc.
sudo apt upgrade --> install the new versions. (If a new linux kernel is installed, you'll need to reboot)
sudo reboot --> reboots the remote computer
sudo shutdown --> powers down the remote computer
lscpu ---> check cpu frequency
nvidia-smi ---> check status of nvidia cards

Unfortunately, that's as far as I have gone right now. I notice that several things about this configuration are site (IP) specific, so I can't really do things that way I am used to doing, which is to configure the box on my free time at home, them lug it to work and plug it in. With my old way of doing things, this is no problem at all, since the remote desktop software does not care where the machines are. So, I guess if I want to do it this way, I'll need to lug the box to work, set up a work area with a keyboard and a monitor, and learn Linux after hours at the shop, or know in advance what the IPs are going to be by configuring them all statically, which I don't quite know how to do yet.

 

[Ken g6]

Before I began setting up IP access, I just used SSH forwarding. You can do that on Windows too if you install Cygwin with an X server. (Though it's been years since I've done that.) Then do `ssh -XC boinc_computer boincmgr`.

You'd think it would take more bandwidth, but although it may be less responsive, it seems to need less bandwidth too.

 

[crashtech]

Okay, well, I decided to try a different remote desktop software, and finally got AnyDesk to accept unattended imcoming connections. Linux really doesn't like that, and probably for really good reasons. But I want this to stay fun as a hobby, and I still have the steep Linux learning curve to climb. So, baby steps it shall be. Now that I can access this first remote Linux host in the way that I'm accustomed, I'm going to move on to setting up multiple instances and adding commonly used projects. Then there are all the custom apps, etc. which need to be implemented as well, then apply all of that to the three other headless machines. After that, who knows if I may slowly graduate to a more secure, streamlined, and less resource intensive method of remote access. Certainly to do so will be easier with all the machines in their intended location instead of sitting by my desk at home.

 

[crashtech]

How will I tell if I've successfully started a second BOINC instance, will boinc-client show up twice in "/etc/init.d"?

 

[biodoc]

How will I tell if I've successfully started a second BOINC instance, will boinc-client show up twice in "/etc/init.d"?

I start a second boinc instance manually so it won't show up in /etc/init.d and it won't start up if the computer is rebooted. I check to see if it is running by connecting to it using boinc manager. Under the file menu choose 'select computer' and enter IPaddress colon portnumber and then password. What command did you use to start the second instance?

 

[crashtech]

This morning, I have gotten the second instance to work. But, my new problems:

1. Terrible image artifacts on the Linux desktop when viewed on my main Windows computer. Can't access hardware acceleration properties in Linux.

2. PrimeGrid AP27 doesn't work in Mint with an RTX 2060, where it did work on the same GPU under Windows. Would like to resolve that quickly, obviously.

Edit: Going to explore OpenCL drivers now, perhaps that will help.

 

[biodoc]

Edit: Going to explore OpenCL drivers now, perhaps that will help.

I believe this generic opencl loader is the one that worked for me.

sudo apt install ocl-icd-libopencl1

Also, for FAH I had to install the development libraries:

sudo apt install ocl-icd-opencl-dev

 

[crashtech]

Well the odd thing is that OpenCL seems to be working, just to verify I dl'ed and ran one of the Luxmark benches:

Spoiler

Also clinfo reports this:

Spoiler

ga7pxsl@ga7pxsl-GA-7PXSL:~$ clinfo
Number of platforms                               1
  Platform Name                                   NVIDIA CUDA
  Platform Vendor                                 NVIDIA Corporation
  Platform Version                                OpenCL 1.2 CUDA 10.2.120
  Platform Profile                                FULL_PROFILE
  Platform Extensions                             cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_fp64 cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll cl_nv_copy_opts cl_nv_create_buffer
  Platform Extensions function suffix             NV

  Platform Name                                   NVIDIA CUDA
Number of devices                                 1
  Device Name                                     GeForce RTX 2060
  Device Vendor                                   NVIDIA Corporation
  Device Vendor ID                                0x10de
  Device Version                                  OpenCL 1.2 CUDA
  Driver Version                                  430.26
  Device OpenCL C Version                         OpenCL C 1.2 
  Device Type                                     GPU
  Device Topology (NV)                            PCI-E, 02:00.0
  Device Profile                                  FULL_PROFILE
  Device Available                                Yes
  Compiler Available                              Yes
  Linker Available                                Yes
  Max compute units                               30
  Max clock frequency                             1800MHz
  Compute Capability (NV)                         7.5
  Device Partition                                (core)
    Max number of sub-devices                     1
    Supported partition types                     None
  Max work item dimensions                        3
  Max work item sizes                             1024x1024x64
  Max work group size                             1024
  Preferred work group size multiple              32
  Warp size (NV)                                  32
  Preferred / native vector sizes                 
    char                                                 1 / 1       
    short                                                1 / 1       
    int                                                  1 / 1       
    long                                                 1 / 1       
    half                                                 0 / 0        (n/a)
    float                                                1 / 1       
    double                                               1 / 1        (cl_khr_fp64)
  Half-precision Floating-point support           (n/a)
  Single-precision Floating-point support         (core)
    Denormals                                     Yes
    Infinity and NANs                             Yes
    Round to nearest                              Yes
    Round to zero                                 Yes
    Round to infinity                             Yes
    IEEE754-2008 fused multiply-add               Yes
    Support is emulated in software               No
    Correctly-rounded divide and sqrt operations  Yes
  Double-precision Floating-point support         (cl_khr_fp64)
    Denormals                                     Yes
    Infinity and NANs                             Yes
    Round to nearest                              Yes
    Round to zero                                 Yes
    Round to infinity                             Yes
    IEEE754-2008 fused multiply-add               Yes
    Support is emulated in software               No
  Address bits                                    64, Little-Endian
  Global memory size                              6220218368 (5.793GiB)
  Error Correction support                        No
  Max memory allocation                           1555054592 (1.448GiB)
  Unified memory for Host and Device              No
  Integrated memory (NV)                          No
  Minimum alignment for any data type             128 bytes
  Alignment of base address                       4096 bits (512 bytes)
  Global Memory cache type                        Read/Write
  Global Memory cache size                        491520 (480KiB)
  Global Memory cache line size                   128 bytes
  Image support                                   Yes
    Max number of samplers per kernel             32
    Max size for 1D images from buffer            134217728 pixels
    Max 1D or 2D image array size                 2048 images
    Max 2D image size                             32768x32768 pixels
    Max 3D image size                             16384x16384x16384 pixels
    Max number of read image args                 256
    Max number of write image args                32
  Local memory type                               Local
  Local memory size                               49152 (48KiB)
  Registers per block (NV)                        65536
  Max number of constant args                     9
  Max constant buffer size                        65536 (64KiB)
  Max size of kernel argument                     4352 (4.25KiB)
  Queue properties                                
    Out-of-order execution                        Yes
    Profiling                                     Yes
  Prefer user sync for interop                    No
  Profiling timer resolution                      1000ns
  Execution capabilities                          
    Run OpenCL kernels                            Yes
    Run native kernels                            No
    Kernel execution timeout (NV)                 Yes
  Concurrent copy and kernel execution (NV)       Yes
    Number of async copy engines                  3
  printf() buffer size                            1048576 (1024KiB)
  Built-in kernels                                
  Device Extensions                               cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_fp64 cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll cl_nv_copy_opts cl_nv_create_buffer

NULL platform behavior
  clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...)  NVIDIA CUDA
  clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...)   Success [NV]
  clCreateContext(NULL, ...) [default]            Success [NV]
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_DEFAULT)  No platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU)  No platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM)  Invalid device type for platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL)  No platform

ICD loader properties
  ICD loader Name                                 OpenCL ICD Loader
  ICD loader Vendor                               OCL Icd free software
  ICD loader Version                              2.2.11
  ICD loader Profile                              OpenCL 2.1
ga7pxsl@ga7pxsl-GA-7PXSL:~$

But I am going to go ahead and do what you suggest, if you think it might be helpful.

 

[biodoc]

As I recall, the Nvidia opencl loader doesn't work on some projects, so installing the generic loader solved that problem.

EDIT: it looks as though you already have the generic icd loader installed.

 

[crashtech]

Hmm, well it's working now. The sudo apt install ocl-icd-libopencl1 command returned that the latest version was already installed, but sudo apt install ocl-icd-opencl-dev did install, and now it looks like its working. Also the AnyDesk artifacting is gone, but I had been playing around with more than one thing, so can't attribute the cure to anything in particular.

Next is to get Boinctasks to see the second instance.

 

[biodoc]

Yeah it looks like one of the opencl dev libraries is required for running the ap27 app:

mark@x16-linux2:/var/lib/boinc/projects/www.primegrid.com$ ldd ap27_2.6_opencl_linux64
linux-vdso.so.1 (0x00007fff881e5000)
libOpenCL.so.1 => /usr/lib/x86_64-linux-gnu/libOpenCL.so.1 (0x00007fed3ba29000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fed3b80a000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fed3b46c000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fed3b07b000)
/lib64/ld-linux-x86-64.so.2 (0x00007fed3bf6e000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fed3ae77000)

 

[crashtech]

Another example of why it's so hard for non-Linux guys like me to get things running. I did not see any documentation in regards to needing the dev libraries. This also brings back to mind the viability of simply cloning this install for use on the other machines, which are probably similar enough to make such a thing work. Would need to edit some relevant things, hostname immediately comes to mind.