GSWL Part 2: Compiling Dependencies for Bitcoin Based Cryptocurrencies
Table of Contents
Part 2 of GSWL is fairly more complicated than part 1. We are looking at how to prep our Linux install for compiling various Bitcoin based cryptocurrencies from source code. That means that we are heading into rabbit hole that goes farther back than 2011. If for nothing else, you need to know how to compile the Berkeley4.8db independently. Hopefully my directions on that below save you from pulling your hair out.
New Terminal Commands
The "cd" Command
"cd" is short for change directory, and its the only method to navigate between folders in almost any command line. Despite being so common, cd is a bit complicated at first. To fully understand how cd works, we need to examine how file systems work. We can look at our file system like a pyramid, at the top of the pyramid you find whats called your "root" directory. In Linux (and most operating systems) it is expressed as "/", thats it. This directory most commonly requires significant privledges to add files to, and hardly ever has files added to it directly due to security concerns. Below that level we have a few levels that run simultaniously: user containers, OS containers, and shared libraries/software containers.
These are all exactly what you would expect them to be. The user container contains all the data specific to a specific user, often each user has their own container. The location for the user container that is logged in to any system based on Unix is almost always "~". The OS container holds all the core operating system files. Often this container is protected as well to prevent security issues. When you install a piece of software you usually install it to the shared software container or the user container. Almost never do you install to the OS container itself.
Finally the shared software/libraries container is where all the things that users install to the computer, and not to their specific user account, get installed. It is accessible to all user accounts on a computer, not just the user logged in at the time. Below these levels we get into all kinds of smaller and much more multitudinous folders and files. Really we don't need to bother with knowing that part of the analogy.
The important thing to know is that we are moving up and down in the system. Up is toward the "root" or "/" location, and down is toward specific files. When you open a terminal window in Linux, you are usually taken to the default location, or "~". This location, "~" is the top of your user container. Here is where commands that download files will default to download them to. They wont default to downloading them to "/" nor should they, as that could put your computers security at risk. "~" can be looked at as the top of a pyramid that just contains your stuff. Its technically found in Ubuntu at /home/USERNAME/, but in terminal windows it will display as "~". To mitigate risk, you should never use sudo with any command that downloads a remote file other than verified repositories.
"~" and "/"
To move between "~" and "/" all you need to do is either use cd / or cd ~. You will see the active directory (to the left of the $ in terminal) update as you move around. To go up a folder simply use cd .. and to go down a folder use cd FOLDERNAME. You can also navigate to places from a distance by typing in the full folder location like cd /usr/local/bin or cd ~/Desktop/Litecoin. Its important to note that capitalization is read by cd, so ~/desktop is not the same as ~/Desktop.
The "LS" command
As important as cd is, it doesn't help anyone at all without the ls command. ls is the command to simply list all the files within the directory. This is really helpful when combined with the cd command as it allows you to find the name of directories within the folder you are in.
The "make" command
Make is used to compile source code into binaries. Usually you need to run a configuration script, or an autogen script prior to running make. Once you run make, you then need to install the binaries by running make install. Make is folder specific, so if you move folders between running make and make install, you will have to go back to the original folder to get make install to work. Also, you can control how many simultaneous build tasks make runs by using the -j modifier(i.e. make -j 2). The number for -j should never exceed your number of cores minus one.
The ". /" Command
"./" is less of a command, and more of an instruction. Basically you use ./ before a script or binary to make the script or binary execute. So if you have a binary named "litecoin-qt", you would run it from terminal by entering ./litecoin-qt. Running the command as ../ will let you specify a folder at the same level as the one you are in and then a script within it to run. This comes in handy when compiling, and we will see it within the Berkeley4.8db compilation instructions.
The "wget" Command
wget is a command that allows us to download a file to the active directory. It has it's uses alongside git and other download commands, we will use it when we compile our local copy of the Berkeley4.8db.
The "unzip" Command
unzip does what it says it does, it unzips a compressed file. With no modifiers, it will extract all of the contents of the specified file to the current active directory.
The "git" Command
The command git comes to us through a new package we will install later on in this tutorial. Essentially git allows us to download whole repositories from Github, instead of just a single file like wget.
Basic Dependencies for Cryptocurrencies
For this part of GSWL we will be working with a brand new installation of Ubuntu 20.04.1 LTS on a Parallels VM run on a MacBook Pro. Your mileage on any other system, distro, or VM platform may vary. We will be walking through how to compile from source the Litecoin wallet. This process should get you about 99% of the dependencies you need to compile other Altcoins on your systems.
Software to Install from the Software Package
There are a few things we will want to go ahead and grab while we are in the Software application. Start your search by clicking on the magnifying glass icon in the upper left hand corner.
Sublime Text is a text editor program. It will come in handy in a lot of ways, but mostly for average users will be good for creating .conf files for miners or other packages that require them.
Dependencies to Download From Terminal for Cryptocurrencies
There are quite a few dependencies you will need to grab from terminal to get ready to compile Litecoin. The ones below are all you need to compile Litecoin. However, you will likely need to add others with different Altcoins. I'll show you how to find those needs as we go through the process.
Libssl is a library that contains a random number generator as well as some other cryptographic tools. This is obviously important as we are dealing with Cryptocurrencies and cryptography is the core of this technology.
$ sudo apt install libssl-dev
Libboost is a library that helps applications structure themselves correctly. A quick word of warning, there are a lot of versions of libboost out there. Some Altcoins are built correctly and look for the most recent version of boost. Some pre-built binaries are not built correctly and are looking for older versions. This is part of the reason that everyone should know how to compile from source. It prevents this kind of nonsense with versions of dependencies. That said, there are some dependencies that are globally version specific and have to be found in weird ways.
$ sudo apt install libboost1.71-all-dev
Libevent is a library for allowing compiled software to communicate with network devices. It is most likely already installed on your computer already, but if not you'll want to use the command below to install it.
$ sudo apt install libevent-dev
Miniupnpc allows for applications to support UPnP to help navigate firewalls.
$ sudo apt install libminiupnpc-dev
This is the infamous Berkeley4.8db that is used by every Altcoin that is based on Bitcoin. It is not available using Aptitude, nor can you obtain it by using the common directions of adding the Bitcoin PPA. Instead, you'll need to follow the second set of instructions from Crypto and Coffee that I have copied below.
$ wget http://download.oracle.com/berkeley-db/db-4.8.30.zip
$ unzip db-4.8.30.zip
$ cd db-4.8.30
$ cd build_unix
$ ../dist/configure --prefix=/usr/local --enable-cxx
$ sudo make
$ sudo make install
If you happen to follow the first set on that site, you will be greeted by the error below, and then have to go into the Software & Updates program to remove the PPA from the Software Sources tab, lest you want to see that error every time you run apt update or apt-get update.
However, even this second set will fail to make properly, which leads us to another blog (fsanmartin.co), with another fix. This time, we will need to use Sublime Text to make the fix.
Open up Sublime text and select open file from the file menu. Then find the file db-4.8.30.NC/dbinc/atomic.h by selecting "Home" from the quick locations on the left side of the file browser, and then db-4.8.30.NC then dbinc, then atomic.h
Sublime Text will automatically show you the line number, so all you have to do is go to line 147 and find: __atomic_compare_exchange((p), (o), (n)) and replace it with __atomic_compare_exchange_db((p), (o), (n)). Then head to line 179 and find static inline int __atomic_compare_exchange( and replace it with static inline int __atomic_compare_exchange_db(. Once you do this, save the file and you should be able to run the sequence of commands from Crypto and Coffee above without issue.
Note: You can run the "make" command with the modifier -j # and it will run multiple tasks at a time based on how many processor cores you specify. For example if you have a dual-core processor, you can run make twice as fast by running sudo make -j 2.
Qt is a GUI (graphical user interface) toolkit for building out GUI's for software binaries. There are actually a couple qt packages we will want to install so we will just download the base package which should put all of them on our computer for us.
$ sudo apt install qtbase5-dev
Protobuf is short for protocol buffers. These are used in the context of cryptocurrencies as a way to run transactions from the GUI. There are also a million versions of Protobuf out there. We again will just go for the -dev variant of this to solve any issues.
$ sudo apt install libprotobuf-dev
This library allows for the GUI to encode QR codes that you can share to receive or send payments with.
$ sudo apt install libqrencode-dev
This has to do with JSON parsing and encoding, its optional, but go ahead and grab it.
$ sudo apt install libunivalue-dev
Again optional, this library has to do with ZMQ and its features within a Bitcoin based client.
$ sudo apt install libzmq3-dev
Git is a terminal command tool that allows us to download repositories of files off the internet. You will use it a lot unless you are more fond of downloading and extracting compressed files and then cleaning all that up on your own.
$ sudo apt install git
This package group includes a lot of packages and libraries that give us the ability to compile GUI elements for bitcoin based altcoins.
$ sudo apt install qttools5-dev-tools
Here we go again, with another protocol buffer package. Like I said earlier, there are a ton of these, and this one specifically is needed to push the configure script with GUI later on.
$ sudo apt install protobuf-compiler
Download the Source Code
Alright, so now that we have all the dependencies we need for Litecoin, lets go ahead and grab the sourcecode from Github. Quickly though, lets go ahead and run both sudo apt update and sudo apt-get update just to make sure all of our stuff is up to date after installing all those packages. Then we will use a handy bit of terminal commands to copy the Github repository straight to our harddrive.
$ git clone https://github.com/litecoin-project/litecoin.git
Using the git command with clone modifier, we can then supply a URL to copy an entire Github repository can be downloaded and extraxted as it should be to your harddrive. This saves us from having to go the Github page, click download source, then extract the zip or other compressed file to the location we want it at. Once that command finishes running, we will want to move into the directory it was copied into using the cd command.
$ cd litecoin/
Start the Configure Process
Now that we are in the Litecoin folder, and we have all of our dependencies downloaded, we are ready to go ahead and start the actual process of compiling the source code. All we need to do to get this started is run the following command in terminal.
After running autogen.sh we are ready to configure the compilation. This, nine times out of ten, will be where you get errors for not having dependencies installed. Its also where you can run the command with modifiers to change the way the final binary comes out. Go ahead and run the following command:
What you see print out will be a long list of the program looking at the dependency libraries you have installed on your computer, and then telling you the way that the program is configured at the end. The important part will look something like this:
Options used to compile and link:
with wallet = yes
with gui / qt = no
with zmq = yes
with test = yes
with fuzz = no
with bench = yes
with upnp = yes
use asm = yes
scrypt sse2 = no
debug enabled = no
gprof enabled = no
werror = no
target os = linux
build os =
So as you can see, the configuration is set up to not provide us with a GUI wallet. Its also going to include a test and bench version of the binary. For most purposes, you wont need the test and bench versions of the binaries, so we can save a bit of time and space by running the configure script with a few modifiers (all of the modifiers available can be found by typing ./configure --help).
$ ./configure --with-gui --disable-tests --disable-bench
This should run the configure script again, and if you have all the dependencies above you should get back to the final readout, this time saying yes to GUI and no to tests and bench.
Compile the Cryptocurrency Wallet
We made it, this is the final moment, you can finally put in the terminal command to tell your computer to start turning all the code into an actual program that you can use. So, get ready, get set, enter:
$ sudo make
Make sure you have a bit of time, it can take anywhere from a few minutes to a couple hours to run make, but once its done the long part of this whole ordeal is over, and all you have to do is install. If you want to watch the printout in the terminal you can, but don't worry if you don't. Any errors that will cause the program not to work will also cause make to stop. You may see a ton of warnings or other things during this process, it happens, usually they don't mean anything. So you will see them as the last error printed out before it stops when you check it later.
Sudo Make Install
Once the make process ends, and as long as it doesn't end with an error, you should be able to run the command sudo make install and have the program copied to your /usr/local/bin folder (most common location). After that you can delete any source code as you only need the binaries that are in the /usr/local/bin/ folder.
Once you run sudo make install and it finishes, head over to /usr/local/bin. You can then launch the litecoin wallet by entering the command ./litecoin-qt.
If the GSWL series helped you out with getting set up compiling binaries from source in Linux, I hope you would consider donating to support this blog. In the future I will be writing about how to find the newest cryptocurrency projects on the internet, and get mining on them quickly. For now feel free to share this blog with anyone whom it may help.