Networking my Existing Equipment - Problem(x) Solutions

This past week I have been migrating and consolidated my desktop and laptop to my networked attached storage (NAS) device, essentially my private cloud storage. My NAS has been sitting in the closet since I move a few years ago. I have a bunch of code and data spread across on my laptop and desktop in unnecessary duplication and versions. Although my laptop has my most recent work, I wanted to sync up my systems so I could use my desktop as my home primary and my laptop as a mobile workstation. I want to consolidate all my projects to a central location so I could tap into it from either resource.

I have also worked on networking my laptop to my desktop to conduct a local experiment on a mini-standalone cluster to learn and explore Apache Spark as well as general distributed computing over a network (more posts coming on that). The following is a simple network diagram of my current configuration (pictures are of exactly what I am using as well).

Equipment Specs

Both my laptop and desktop are running 64-Bit Linux/Ubuntu 20.04 (Desktop). The Synology NAS runs the Synology OS/UI, DiskStation Manager.

Laptop

My laptop has been my primary machine for the last several months. It serves me great when I’m traveling, but its not my favorite to work.
My laptop computer has the following characteristics:

Memory: 15.2 GB
Processor AMD® A10-8700p Radeon r6, 10 compute cores 4c+6g × 4
- benchmark
Clockspeed: 1.8 GHz
Turbo Speed: 3.2 GHz
Number of Cores: 4
Disk Capacity: 1.0 TB

Desktop

I got my desktop back in 2011, so it is a little dated but it still preforms great. In fact, according to the benchmark tests of the processors (linked below), it still outperforms my laptop in almost every area. It is getting time to update this though and the next step in my current network configuration might be to convert this machine to a dedicated server or get a server with current technology and capacity for multiple processor slots (2-4).

My desktop computer has the following characteristics:

Memory: 9.7 GB
Processor: AMD Phenom iix6 1090t processor x 6
- benchmark
Clockspeed: 3.2 GHz
Turbo Speed: 3.6 GHz
Number of Cores: 6
Disk Capacity: 2.5 TB

Raspberry Pi 3

I currently have two Raspberry Pis on my network as well, but for the purpose of this article, I will not go into this aspect for now.

TRENDnet 8-Port Switch

I have a couple of these in my house to distribute wired connections. They have worked as expected. The switch has the following characteristics:

Ethernet Ports: 8 x Gigabit Ports
Capacity: Each port provides high-speed network connection to devices and a 16Gbps switching capacity allows data traffic to flow smoothly, reducing 4traffic bottlenecks.
Data Transfer Rates:
- Ethernet: 10Mbps (half duplex), 20Mbps (full duplex)
- Fast Ethernet: 100Mbps (half duplex), 200Mbps (full duplex),
- Gigabit: 2000Mbps (full duplex)

Synology DS415+

CPU: Intel Atom Quad Core 2.4GHz w/ Floating Point & Hardware Encryption Engine
Memory: 2GB DDR3
Drive Bays: 4x 3.5″ or 2.5″ SATA2
Quad Core CPU with AES-NI Hardware Encryption Engine
2 LAN with Failover and Link Aggregation Support

Configuring my Synology DS 415+

The instructions here are specific to my setup, although the process shouldn’t deviate much. I’m currently using my Synology DS415+ NAS (Amazon Link) with two 3TB WD Red NAS HDD (Amazon Link). Initially I only got two hard drives for storage but knew I could expand as technology improved and came down in price. ($110 for 3TB WD Red in 2014 vs $85 in 2021). I can get one 6TB WD Red Plus HDD for $146 (Feb 12, 2021). I currently run the drives in RAID 1, which is fine since I know everything is being backed up and I don’t need 3 TB right now. I’m currently only at 17% utilization, so plenty of room.

Products Used

Switch

Ethernet Cables

Hard Drives

Synology

Creating the NFS Shared Folders

The first thing I did was connect my NAS to my network via my TRENDnet 8-port Gigabit switch (Amazon Link) with two Cat6 cables. From there I can connect to the device from either machine with the configured login interface. Once I accessed the Synology user interface, I needed to create a “Data Analytics” Shared Folder to house all my code, graphics and data.

In this section, I will walk through connecting and mounting that Shared Folder (NFS) to one of my Linux/Ubuntu 20.04 computers. The process is the same on both machines.

First, navigate to the Control Panel > Shared Folder.

You can create whatever folders you want or edit the existing folders from this area. Since I have already created the folder “DataAnalytics”, I will edit the NFS permissions. Also from the image below you can see all the other tabs and options you can control from this window.

From The “NFS Permissions” tab you can create and edit rules.

From the window that opens up, I can enter the Hostname/IP for whichever machines that I want to have access to this folder. This is necessary for mounting the folder to your computer. If you don’t do this you will get errors and permission denied statements.

Since I have two machines, I created a rule for each. I could have created one rule to simplify the process but I want to limit access for now.

Connecting and Mounting the NAS

Adding IP and Name to hosts

Now we focus the rest of the post on configuring the NFS shared folders on the desired machine. First, I edited the /etc/hosts file to create a name for the IPs associated with my Synology DS415+. Since it has two ports, I have two IPs. While I’m here editing the file, i also added the IP information for my other machines. For each IP i can assign it a name that allows me to refer to the IP by a given name instead of typing out the IP each time. This is useful later, when I start communicating between my desktop and laptop (SSH).

sudo nano /etc/hosts

The form I used is like the following.

xxx.xxx.xx.xxx   hostname

As you can see I have three addresses on my LAN that I will associated by name.

Mounting a Drive from Synology to Ubuntu Linux

`fstab`

First, I need to add the mount information to the file /ect/fstab before I mount a directory in my NAS to either machine.

The fstab is the Linux systems’s filesytem table, and is designed to simplify the mounting/unmounting on the machine on startup. This information saves you the time from mounting the drive each time you want to connect.

sudo nano /etc/fstab

At the bottom of the file enter the information in the following form (separation by space).

 [Device] [Mount Point] [File_system] [Options] [dump] [fsck order] 
hostname:/location/of/nfs/folder /location/of/mounted/folder nfs defaults 0 0

pxs1:/volumeq1/DataAnalytics /home/bear/DataAnalytics nfs defaults 0 0

Field 1: Device or Share
- psx1:/volume1/DataAnalytics
Field 2: Mount Point or where the device will be located
- /home/bear/DataAnalytics
- If the directory/path does not exist, you need to create it with mkdir beforehand.
Field 3: File System Type
- nfs
Field 4: Options
- defaults = rw, suid, dev, exec, auto, nouser, and async
Field 5: Backup Operations
- 0 = no backup
Field 6: File System Check Order
- 0 = fsck(filesystem check) will not check filesystem

After completing your edits to the file, you can run the following code.

sudo mount -a

For more information on fstab check out this page, or type man fstab on the command line.

Permissions

To test to see if the mounted directory is accessible.

ls -l  DataAnalytics
stat DataAnalytics
touch DataAnalytics/test

If for some reason you get responses like permission denied or an error, you can modify the directories permission setting with the following examples. Please references man chown and man chmod for specific arguments for granting permissions and ownership.

sudo chmod -R 775 ./DataAnalytics/
sudo chown -R linux:linux ./DataAnalytics/

Now that the NFS shared folders are accessible on my machines, I can move on to networking between my laptop and desktop (as well as other devices).

Networking Between Machines

Set up SSH

*** Repeat this for each machine in your cluster ***

Run the following on each node in the desired cluster if you don’t already have this setup yet.

sudo apt install openssh-server
ssh -v localhost

Repeat from above (Optional)

As mentioned in a previous section above, you can give each machine IP address a name in place of entering in the IP address each time.

cat /etc/hosts
sudo nano /etc/hosts

When the editor comes up include the following:

[ip address of each machine] [name/pseudonym]
XXX.XXX.XX.XXX  node0
XXX.XXX.XX.XXX  node1
XXX.XXX.XX.XXX  node...
XXX.XXX.XX.XXX  nodeN

Next, I generated an ssh key (RSA default) to enable quick login to other machines, regardless of which is tasked as the master and worker nodes. This simplifies the ssh login process within my local network. You may prefer additional security measures in your network.

Before moving on, I will also copy my public key to a file called authorized_keys.

ssh-keygen
ls .ssh/
cat .ssh/id_rsa.pub >> authorized_keys

ssh-copy-id user@ip

# or if you assigned a name to an IP address:
# assuming currently on node0 (MASTER NODE)
ssh-copy-id user@node1
ssh-copy-id user@node...
ssh-copy-id user@nodeN

You only need to make sure the master node has the information for each worker node. I made all my machines capable of being a master node. For example if you have six machine nodes (1 master 5 slaves), only the master node configuration should have all size ssh public keys. Each slave node needs a minimum of the master node’s public key and their own public key in the authorized_keys file. You can certainly make all node the same, having all public keys in the authorized_keys file. This option allows you the ability to reconfigure your cluster network without having to come back to this step.

*** Repeat this for each machine in your cluster ***

For more information on the ssh function, please refer to the manual page man ssh for details on arguments and options.

Test the Connection(s)

From any node, I can enter the following in the terminal to ssh into any other machine.

ssh user@ip

# or
ssh user@node1 # whichever node you want to connect to

Test Connection speeds on the network

In this example I’ll use iperf, which is a tool for performing network throughput tests. Install iperf on both machines.

sudo apt install iperf

One of the machines runs the server command:

iperf -s

The other machine(s) run the client command:

iperf -c <server ip>

You can see the results on both ends of the test. For my local throughput test I achieved 941-943 Mbps in each direction (send/receive), which is pretty close to 1Gbps. I’m satisfied.

[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1.10 GBytes   943 Mbits/sec

As a reminder I am using Cat6 ethernet cables and a gigabit switch in my current network configuration. Since Cat6 cables can support up to 10Gbps the limiting factors in my configuration are my attached devices and the switch speed itself. If you have an older, non-gigabit switch, you will experience a lower throughput.

For more information, please reference man iperf for more details on the commands arguments and options.

What’s Next?

I have planned the next several follow-on posts where I will detail the following:

Connecting to a remote database (PostgreSQL) over LAN
Migrating my PostgreSQL database to NAS Device
Getting Started with Apache Spark (local)
Getting Started with Apache Spark (Standalone)
Additional distributed computing, including Raspberry Pi.

The main focus will be on using my currently available resources to get as much as I can from them before progressing in my technology.

Resources:

Network share: Performance differences between NFS & SMB

https://ferhatakgun.com/network-share-performance-differences-between-nfs-smb/