Setup Zcash on Development Workstation


This document is a guide for installing zcashd, both mainnet and testnet, on a development workstation.

Original Located at Github

Preamble

This guide has been generated using notes I have collected from various sources that have been useful to me for setting up full node zcashd server. It is useful to have a full node server installed if one is building an application using the zcash blockchain. It is helpful when the zcashd software is communicating and syncronizing on BOTH the main (mainnet) and test (testnet) networks and the start/stop lifecycle is controlled by the operating system. While there is a great deal of good information resources available for zcashd I hope this recipe/guide can bring much of that together, along with a touch of sysadmin, to lower the barrier of entry for developers wishing to build applications utilizing the zcash blockchain.

License

Copyright (c) 2023 Dennis R. Gesker.
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3
or any later version published by the Free Software Foundation;
with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
A copy of the license is included in the section entitled "GNU

Goals

  • Install zcashd (zcashd)[https://github.com/zcash/zcash]
    • Using the apt.z.cash repository
  • Configure zcashd to automatically start using systemd
    • Mainnet (mainnet)
    • Testnet (testnet)
  • Configure zcashd
    • Mainnet (mainnet)
      • RPC Credentials
      • Backup Directory
        • Backup Wallet
        • Generate Pass Phrase
      • Mining Address
      • Connect to Specific upstream zcashd server (optional)
    • Testnet (testnet)
      • RPC Credentials
      • Backup Directory
        • Backup Wallet
        • Generate Pass Phrase
      • Mining Address
      • Connect to Specific upstream zcashd server (optional)
  • Configure logrotated
  • Symlink logs to /var/log directory

Non-Goals

  • Guarantee Security of the configuration
  • This is for a personal workstation only!

Miscellaneous

  • I am not part of the (zcash)[https://github.com/zcash/zcash] project
  • I do use this configuration on my own workstation
    • Use at your own risk!
  • Pull requests to improve these notes are welcome
  • DO NOT USE ANY OF THE GENERATED CREDENTIALS OR ADDRESSES FOUND IN THIS DOCUMENT
    • I will show you how to generate your own credentials using the script provided by the zcash project.

Requirements

  • Debian 11 (Bullseye)
  • Ability to sudo on the system
  • Ability to use a text editor and navigate directories
  • About 400 GB of disk space in your home ~/ directory

Step 1 – Install some prerequisites

sudo apt update
sudo apt install git python logrotate curl 

Step 2 – Install Zcashd (Using apt)

Add an entry for zcash to your apt sources

sudo vi /etc/apt/sources.list

Append the following line:

deb [arch=amd64 signed-by=/usr/share/keyrings/zcash.gpg] https://apt.z.cash/ bullseye main

A reference copy of a sources.list file can be found here.

Add the zcash project apt repository gpg keys:

sudo wget -qO - https://apt.z.cash/zcash.asc | sudo gpg --import
sudo gpg --output /usr/share/keyrings/zcash.gpg --export 6DEF3BAF272766C0

Tip: Always verify any gpg keys provided to you in guides and scripts. Compare the signature 6DEF3BAF272766C0 to the signature published in the keyserver and in the official documentation provided by the zcash project.

Update apt and install zcashd

sudo apt update
sudo apt install zcashd

The zcashd software is now installed on your system. However, we are going to complete its configuration before we attempt to start the daemon.

Step 3 – Download Source Code and Parameters

We have installed the zcash software using the apt repository. Why are we downloading the source code?

Good question! There is a command, a python 2 script, that we require that does not seem to be included with the Debian package provided by the zcash project. We will use that script when we generate credentials in a step of this guide below. There are other tools available that will also let us generate these credentials but since this guide is about installing zcashd we will use the scripts made available by the zcash project.

mkdir ~/Development/
cd ~/Development/
./zcutil/fetch-params.sh
git clone git@github.com:zcash/zcash.git

We will also install the zcash blockchain parameters which will be installed to ~/.zcash-params.

zcash-fetch-params

Step 3 – Add Zcashd configuration(s)

Create ~/.zcash directory parameters and copy both zcash.conf.mainnet and zcash.conf.testnet into the ~/.zcash folder.

Also, create a symlink for one of these configuration files to ~/.zcash/zcash.conf. This zcash.conf file is not needed specifically needed for this recipe but zcash will complain and fail to start should you launch zcashd manually and do not specify a configuration file on the command line. So, it is good to have a default zcash.conf file in the expected place.

mkdir ~/.zcash
cp zcash.conf.mainnet ~/.zcash
cp zcash.conf.testnet ~/.zcash
ln -s ~/.zcash/zcash.conf.mainnet ~/.zcash/zcash.conf

RPC Credentials

There are a few lines in zcash.conf.mainnet and zcash.conf.testnet that will need your SPECIAL attention.

  • rpcauth=mySecureUsername:mySecureCredential
  • rpcuser=mySecureUsername
  • rpcpassword=mySecurePassword

Even though this is a setup for your development workstation it is still wise to take security seriously and use good secure credentials.

NoteThis guide does NOT cover TLS configuration!

‘rpcuser’ and ‘rpcpassword’ are being deprecated. Zcashd will issue a log entry confirming this. Still, it seems that if you want to make sue of another package from the zcash project [lightwalletd][https://github.com/zcash/lightwalletd] these configuration entries need to be present.

‘rpcauth’ will correspond to the credentials one would put into the header of the REST calls that your application makes to the zcashd server.

Please use a much better username than mySecureUsername.

We’ll use a utility included with the zcash source code to generate the values you will need to replace the mySecurePassword and mySecureCredential values in your zcash.conf.mainnet and zcash.conf.testnet files.

~/Development/zcash/share/rpcuser/rpcuser.py mySecureUsername

The output will look something like:

String to be appended to zcash.conf:
rpcauth=mySecureUsername:1228c4e1b813a679f35ea18cfe81b8d$70b7a9a81583cd612dfd68d1a6a074feef90dff106eabed799f253762d9ac2f6
Your password:
SQ-VFegQi-IJFQYvtXvzhaP1HQOcaGfhskZ2aPa_M1M=

So, In this example you would update the lines:

rpcauth=mySecureUsername:mySecureCredential
rpcuser=mySecureUsername
rpcpassword=mySecurePassword

to

rpcauth=mySecureUsername:1228c4e1b813a679f35ea18cfe81b8d$70b7a9a81583cd612dfd68d1a6a074feef90dff106eabed799f253762d9ac2f6
rpcuser=mySecureUsername
rpcpassword=SQ-VFegQi-IJFQYvtXvzhaP1HQOcaGfhskZ2aPa_M1M=

Again, use the values you generated for ‘rpcauth’ and ‘rpcpassword’ in both the zcash.conf.mainnet and zcash.conf.testnet files in your own ~/.zcash folder and choose a better ‘rpcuser’ value/name.

Backup Directory

Currently ‘backupdir’ is set to /tmp which is not at all secure. It is important that you update this value in both files.

Make sure ‘backupdir’ in both the zcash.conf.mainnet and zcash.conf.testnet are set to VALID and SAFE locations.

Mining address

If you set ‘gen=1’ in the zcash.conf.mainnet and zcash.conf.testnet update the ‘mineraddress’ to your OWN zcash addresses! And, it is best if you have both an mainnet and testnet address NOT associated with this wallet.dat file. The reason being that sometimes in development things go wrong. Even though it is hard to do you may have a scenario where the wallet.dat file gets corrupted and, wouldn’t you know it, you don’t have a way to restore the wallet.dat file from backup some reason. Hey! Thing happen and go wrong. In this guide we are going to backup both our mainnet and testnet wallet.dat files but still consider having a second address for any mining activity performed by your zcashd nodes and use those address(es) in the nodes that you control.

You will see that generating a new wallet or address is pretty trivial after you setup up your first zcash node.

Connect

Both zcash.conf.mainnet and zcash.conf.testnet have a commented out ‘connect’ entry. If you have the resources to run a full node on your LAN/WAN that can safely be connected to the Internet; NOT YOUR WORKSTATION AS WE ARE SETTING UP HERE. Having more nodes in a blockchain network is always helpful to the blockchain network you are using. And, if this ISOLATED and PROPERLY FIRE WALLED dedicated full node is also accessible on your LAN then you can use the IP address of that dedicated node as your connect entry. Then your workstation will connect ONLY to the dedicated node on your LAN instead of random nodes on the Internet.

Step 3 – Add Zcashd Systemd configuration(s)

In this repository there are two systemd configuration files; zcashd_mainnet.service and zcashd_testnet.service. Systemd is the part of your Debian system that, among other things, controls which services and daemons run on your system.

In both of these files update the ‘ExecStart’ lines to be sure they are pointing at YOUR home directory and more specifically the corresponding zcash.conf.????net file under in the ~/.zcash directory.

In zcashd_mainnet.service:

ExecStart=/usr/bin/zcashd -conf=/home/yourUsername/.zcash/zcash.conf.mainnet

In zcashd_testnet.service:

ExecStart=/usr/bin/zcashd -conf=/home/yourUsername/.zcash/zcash.conf.testnet

and copy these files to your systemd folder and enable these as systemd services.

sudo cp zcashd_mainnet.service /etc/systemd/system/
sudo cp zcashd_testnet.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable zcashd_mainnet
sudo systemctl enable zcashd_testnet

The zcashd server is now configured so that it can be started or stopped on either the mainnet or testnet just like any other service on your Debian workstation. The ‘sudo systemctl enable’ commands we issue instruct systemd to start zcashd when you reboot.

Step 3 – Add ZcashD logrotate setup

Having the log files generated by zcashd stored and rotated along with the rest of your system logs is useful especially if you leave your workstation running for a long time.

Prepare a location for the logs

We will prepare a location for the logs and create symlinks to the default locations:

sudo mkdir /var/log/zcashd
sudo chown -R yourUsername:yourUsername /var/log/zcashd
touch /var/log/zcashd/db.mainnet.log
touch /var/log/zcashd/debug.mainnet.log
touch /var/log/zcashd/db.testnet.log
touch /var/log/zcashd/debug.testnet.log
ln -s /var/log/zcashd/db.mainet.log ~/.zcash/db.log
ln -s /var/log/zcashd/debug.mainet.log ~/.zcash/debug.log
mkdir ~/.zcash/testnet3
ln -s /var/log/zcashd/debug.testnet.log ~/.zcash/testnet3/debug.log
ln -s /var/log/zcashd/db.testnet.log ~/.zcash/testnet3/db.log

Note: We could have added a ‘debuglogfile’ entry to our zcash.conf.mainnet and zcash.conf.testnet files but I prefer to use symlinks on my workstation.

Add a logrotate configuration

Copy over the provided logrotate files.

sudo cp zcashd.logrotate /etc/logrotate.d/zcashd

Step 4 – Start Zcashd

Earlier we enabled zcashd as a systemd services for mainnet and testnet. We are going to start those services now!

Mainnet

Let’s start zcashd on mainnet first:

sudo systemctl start zcashd_mainnet
tail -f ~/.zcash/debug.log

You will now see the log files being generated by the zcashd server for mainnet. ‘Ctrl-c’ to exit logging.

Now what is required is patience! It could takes a very long time, days even, to fully syncronize the mainnet blockchain. Most commands, including RPC commands, will not be available until the mainnet blockchain is fully syncronized.

Testnet

Repeat for testnet. Notice that testnet files are in a ~/.zcash/testnet3; a subdirectory of ~/.zcash

Note: You CAN start zcashd on testnet BEFORE mainnet has finished syncronizing.

sudo systemctl start zcashd_testnet
tail -f ~/.zcash/testnet3/debug.log

Even though the testnet blockchain is much smaller than the mainnet blockchain it can still take quite a while to syncronize the testnet blockchain. Most commands, including RPC commands, will not be available until the testnet blockchain is fully syncronized. Again, patience.

Step 4 – Zcashd Files

With the blockchain(s) fully syncronized one will see a number of new files and folders created under ~/.zcash. Let’s take a look!

Log Files

With both zcashd configurations running your ~./zcash/debug.log and ~./zcash/testnet3/debug.log files should be filling.

You can watch these log files being appended in real time which comes in handy when you use more verbose debug settings in your configuration, e.g. ‘debug=all’.

File Structure

Your directory tree will look something like the tree structure below except that you will have MANY MANY dat and ldb files.

/home/yourUsername/.zcash
├── banlist.dat
├── blocks
│   ├── blk00000.dat
│   ├── ...
│   ├── index
│   │   ├── 000123.ldb
│   │   ├── ...
│   │   ├── CURRENT
│   │   ├── LOCK
│   │   ├── LOG
│   │   ├── LOG.old
│   │   └── MANIFEST-405383
│   ├── rev00000.dat
│   ├── ...
├── chainstate
│   ├── 000123.ldb
│   ├── ...
│   ├── CURRENT
│   ├── LOCK
│   ├── LOG
│   ├── LOG.old
│   └── MANIFEST-1093374
├── database
│   └── log.0000000123
├── db.log -> /var/log/zcashd/db.mainnet.log
├── debug.log -> /var/log/zcashd/debug.mainnet.log
├── fee_estimates.dat
├── peers.dat
├── testnet3
│   ├── banlist.dat
│   ├── blocks
│   │   ├── blk00000.dat
│   │   ├── ...
│   │   ├── index
│   │   │   ├── 000123.ldb
│   │   │   ├── ...
│   │   │   ├── CURRENT
│   │   │   ├── LOCK
│   │   │   ├── LOG
│   │   │   ├── LOG.old
│   │   │   └── MANIFEST-041811
│   │   ├── rev00000.dat
│   │   ├── ...
│   ├── chainstate
│   │   ├── 000123.ldb
│   │   ├── ...
│   │   ├── CURRENT
│   │   ├── LOCK
│   │   ├── LOG
│   │   ├── LOG.old
│   │   └── MANIFEST-005528
│   ├── database
│   │   └── log.0000012345
│   │   └── ...
│   ├── db.log -> /var/log/zcashd/db.testnet.log
│   ├── debug.log -> /var/log/zcashd/debug.testnet.log
│   ├── fee_estimates.dat
│   ├── peers.dat
│   ├── testnet3
│   │   ├── banlist.dat
│   │   ├── blocks
│   │   │   ├── blk00000.dat
│   │   │   ├── ...
│   │   │   ├── index
│   │   │   │   ├── 000123.ldb
│   │   │   │   ├── ...
│   │   │   │   ├── CURRENT
│   │   │   │   ├── LOCK
│   │   │   │   ├── LOG
│   │   │   │   ├── LOG.old
│   │   │   │   └── MANIFEST-032489
│   │   │   ├── rev00000.dat
│   │   │   ├── ...
│   │   ├── chainstate
│   │   │   ├── 000123.ldb
│   │   │   ├── ...
│   │   │   ├── CURRENT
│   │   │   ├── LOCK
│   │   │   ├── LOG
│   │   │   ├── LOG.old
│   │   │   └── MANIFEST-004291
│   │   ├── database
│   │   │   └── log.0000000123
│   │   ├── fee_estimates.dat
│   │   ├── peers.dat
│   │   └── zcashd.pid
│   ├── wallet.dat
│   └── zcashd_testnet.pid
├── wallet.dat
├── zcash -> .zcash
├── zcash.conf -> zcash.conf.mainnet
├── zcash.conf.mainnet
├── zcash.conf.testnet
└── zcash_mainnet.pid

Wallet Files

Of all the files found under ~./zcash the MOST important are your wallet.dat files. These files, and their contents, must be protected!

/home/yourUsername/.zcash
├── testnet3
│   ├── wallet.dat
├── wallet.dat

Wallet File Discussion

These data can files can be protected in a number of ways and it is best to do them all!

  • Backup these files themselves
  • Copy to a safe external media
  • Copy to a SECURE network location
  • Issue the backup command to create a zcashd backup file
    • Copy that generated HUMAN READABLE file to external media
    • Copy that generated HUMAN READABLE file to a SECURE network location
    • Issue a backup command EVERY TIME you create a new address
      • The more often the better!
      • The backup file is put into the directory associated with the ‘backupdir’ setting.
        • Double check that ‘backupdir’ points to a secure directory!!
  • Create a ‘Recovery Phrase’
    • Store this recovery phrase someplace safe

We’re going to address some of these safety/security items in a subsequent section.

Note: If you move your wallet.dat files OUT of their current locations and restart the zcashd_mainnet and zcashd_testnet services the zcashd server will create NEW wallet.dat files!

Look at the Wallet

Look at the mainnet wallet first

zcash-cli getwalletinfo 
zcash-cli listaddresses

The output of these commands will look something like

{
  "walletversion": 60000,
  "balance": 0.00000000,
  "unconfirmed_balance": 0.00000000,
  "immature_balance": 0.00000000,
  "shielded_balance": "0.00",
  "shielded_unconfirmed_balance": "0.00",
  "txcount": 0,
  "keypoololdest": 1675120114,
  "keypoolsize": 101,
  "paytxfee": 0.00000000,
  "mnemonic_seedfp": "****************************************************************" **REDACTED**
}
[
  {
    "source": "mnemonic_seed",
    "transparent": {
      "addresses": [
        "t1*********************************" **REDACTED**
      ]
    }
  }
]

You will see some information about the wallet including the “mnemonic seed” and the single TRANSPARENT address in the wallet.

Try to avoid using your ‘transparent’ or ‘T’ address as transactions are visible. As a matter of fact avoid send OR receiving to ‘transparent’ addresses. Still, there might be a circumstance where you need a transparent address so you have one.

Wallet Addresses

We are going to add TWO new additional addresses.

The first will be a ‘Z’ address which is an encrypted address meaning to/from other encrypted ‘Z’ addresses are private.

zcash-cli z_getnewaddress

The second will be a ‘U’ address which is a ‘unifed’ address.

‘U’ addresses are future proof. As the zcash blockchain evolves and ‘unifies’ transparent and subsequent types of encrypted transactions the ‘U’ addresses should prove to be resilient as new technologies are introduced. These new ‘U’ addresses should prove very handy in simplifying the transparent/encrypted complexity form the end user of your application as the zcash blockchain ecosystem goes through inevitable evolution. It is a very activy project after all.

‘U’ addresses appear to be the future of the zcash addressing scheme and if you are doing development against the zcash blockchain I recommend getting familiar with the ins and outs of sending transactions using a unified address. Especially user feedback on when transactions could be attempting to utilize one or more TRANSPARENT utxos which could result in a transaction being public!

Note: The ‘z_sendmany’ RPC method has a ‘privacyPolicy’ parameter that seems to work well with ‘U’ addresses to help prevent mistakes.

Unified Addresses are explained here and here and here and are beyond the scope of this guide.

zcash-cli z_getnewaccount
zcash-cli z_getaddressforaccount 0

Let’s view the wallet AFTER we have added a new ‘Z’ and ‘U’ address.

zcash-cli listaddresses

The output of the addresses stored in the wallet now looks more like

[
  {
    "source": "mnemonic_seed",
    "transparent": {
      "addresses": [
        "t1RBPsGfbyBEpBJLsjha1ryPbh5swo1h7YU"
        "t**********************************" **REDACTED**
      ]
    },
    "sapling": [
      {
        "zip32KeyPath": "m/32'/133'/2147483647'/0'",
        "addresses": [
          "zs****************************************************************************" **REDACTED**
        ]
      }
    ],
    "unified": [
      {
        "account": 0,
        "seedfp": "****************************************************************", **REDACTED**
        "addresses": [
          {
            "diversifier_index": 0,
            "receiver_types": [
              "p2pkh",
              "sapling",
              "orchard"
            ],
            "address": "u1*******************************************************************************************************************************************************************************************************************" **REDACTED**
          }
        ]
      }
    ]
  }
]


You will see you now have a T (transparent)Z (encrypted), and U (unifed) address. Or, put another way… our wallet.dat files has an address that represents the past, present and future proof addresses of the zcash blockchain.

Now repeat the above for testnet using port 18232 and send some RPC commands using curl with a json payload. This should also provide some hints for when you integrate zcash with your application using REST calls.

curl --user mySecureUsername --data-binary '{"jsonrpc": "1.0", "id":"curltest1", "method": "z_getnewaddress", "params": [] }' -H 'content-type: text/plain;' http://127.0.0.1:18232/;
curl --user mySecureUsername --data-binary '{"jsonrpc": "1.0", "id":"curltest2", "method": "z_getnewaccount", "params": [] }' -H 'content-type: text/plain;' http://127.0.0.1:18232/;
curl --user mySecureUsername --data-binary '{"jsonrpc": "1.0", "id":"curltest3", "method": "z_getaddressforaccount", "params": [0] }' -H 'content-type: text/plain;' http://127.0.0.1:18232/;
curl --user mySecureUsername --data-binary '{"jsonrpc": "1.0", "id":"curltest1", "method": "z_getwalletinfo", "params": [] }' -H 'content-type: text/plain;' http://127.0.0.1:18232/;
curl --user mySecureUsername --data-binary '{"jsonrpc": "1.0", "id":"curltest1", "method": "listaddresses", "params": [] }' -H 'content-type: text/plain;' http://127.0.0.1:18232/;

You can see that ‘method’ section of the json contains exactly the same command/options that we used with the zcash-cli tool.

Step 5 – Wallet Safety

As discussed above it is important to protect the wallet.dat files. Backup up these files as you would backup any other important file on your system. If your wallet.dat file is on mainnet and contains currency BE EXTRA diligent in taking precautions.

I’d recommend backing up using the RPC command (or zcash-cli) anytime you add a new address – AT THE VERY LEAST.

Zcashd Backup Command

The ‘walletrequirebackup’ option is set to FALSE in our zcash.conf.mainnet and zcash.conf.testnet this option should be set to TRUE. Let’s backup our wallet.dat file, set this option to TRUE and restart our mainnet service.

On your development machine I recommend the naming convention wallet[Network][YYMMDD] so for 2021-Jan-01 I would enter:

zcash-cli backupwallet walletMainnet20210101; 

which is the same as

curl --user myusername --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "z_exportwallet", "params": ["walletMainnet20210101"] }' -H 'content-type: text/plain;' http://127.0.0.1:8232/

we can adapt this for testnet by changing the port (and changing the backup file name using Testnet)

url --user myusername --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "z_exportwallet", "params": ["walletTestnet20210101"] }' -H 'content-type: text/plain;' http://127.0.0.1:18232/

Zcashd Recover Phrase

To obtain a recovery phrase we will use a tool that was installed when along with the zcashd server; zcashd-wallet-tool.

Simply issue the command and follow the instructions:

zcashd-wallet-tool

COPY THIS PASS PHRASE TO A SAFE PLACE!

Final Thoughts and Discussion

Dedicated Full Node

Above I mention that is useful to also have a dedicated full node on your LAN. Most of this guide could be helpful but there could/should be some differences I would consider.

  • Apply a really good harding to the operating system even BEFORE attempting to configure zcash!!!
  • The firewall configuration would BLOCK the rpc ports facing the INTERNET; 8232 and 18232
  • The firewall configuration would ALLOW the rpc ports facing the LAN; 8232 and 18232
  • The rpcuser, rpcpassword and rpcauth would be unique to that system

Even better… NO RPC AT ALL!!!

  • The wallet.dat files on BOTH mainnet and testnet would contain NO coins and the files/addresses would be considered disposable
    • Still backed up just in case!
    • zcashd-wallet-tool run against both wallets just in case!
  • All of this would be setup using a regular group also called zcash
    • groupadd -g 2001 zcash
  • All of this would be setup using a regular user called zcash; no login allowed!
    • useradd zcash -u 2001 -g 2001 –no-create-home -s /usr/sbin/nologin
  • TOR would also be used following the directions posted here once version 3 was available
  • Perhaps the configuration files would reside in a subdirectory of /etc
    • /etc/zcashd
    • ‘debuglogfile’ options would point directly to files under /var/log/zcashd/
    • ‘datadir’ would point to a subdirectory of /var
      • /var/zcashd
    • debug off
  • On the workstation(s) on my LAN would use the ‘connect’ option using LAN IP address for this dedicated machine

The biggest benefits would be

  • If/when I blew out or damaged my ~./zcash blockchain data – which I have done – during the course of development activities one could copy down the relevant files/folders to again bootstrap the blockchain syncronization activity on the workstation
    • This cuts the time down from Days to Minutes on a slow connection
  • If you work on team, as I do, more than one workstation can connect to this dedicated server.
    • This can ease the burden on a slow WAN connection quite a bit
  • Supplying a node to the blockchain is always useful if you have the resources

Additional Information Sources

Other Development Environment Alternatives

  • Docker is always nice
    • Zcash Docker
    • A great alternative BUT one still has to deal with blockchain syncronization so a local install was better for my needs.
    • YMMV

This Document

  • Feedback welcome
  • Pull requests welcome
  • Original Located at Github

I hope you found these notes useful, getting zcashd setup relatively quickly and can back to coding your zcash application!

Bare Minimum Debian Hardening Steps

A few absolutely essential hardening steps for a new Debian system.

Preamble

An in NO WAY comprehensive set of notes for hardening a Debian system.

Original Located at Github

License

Copyright (c) 2023 Dennis R. Gesker.
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3
or any later version published by the Free Software Foundation;
with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
A copy of the license is included in the section entitled "GNU

Goals

Provide a QUICK and DIRTY Debian hardening notes.

  • Full Distribution Upgrade
  • Automatic Updates
  • Add REGULAR USER account
  • Generate and Copy ssh key from workstation to new system
  • Disable Password Authentication
  • Install and Enable a Firewall
  • Set System Name
    • Match DNS

Non-Goals

  • Guarantee Security
    • Can’t be done but can be mitigated a bit
  • Provide a substitute for comprehensive system security and administration guides

Requirements

  • Debian 11 (Bullseye)
  • Ability to sudo on the system
  • Ability to use a text editor and navigate directories
  • Assumption: Your main workstation is a Linux
  • Assumption: You have just completed a fresh install of Debian 11 (Bullseye)
    • Server or Virtual Instance

Full System Upgrade

There still may be packages that should be upgraded immediately following a fresh installation. Particularly where the install was performed “offline” and updates were not applied during installation.

apt update;
apt dist-upgrade;
reboot

Automatic Updates

Automatically installing updates published into the STABLE repositories is a good idea. Debian has a package to help you keep your system upto date so install this package and allow it to help you in your efforts to keeping you system up to date.

apt install unattended-upgrades
dpkg-reconfigure --priority=low unattended-upgrades

This package has some additional options. Consider enabling email notifications in your configuration. Check out UnattendedUpgrades at Debian.

Add a REGULAR user account

It is not a good idea to use the ‘root’ account for your everyday needs. The ‘root’ account is best used only for system administration tasks. Create a regular user account and add that account to the ‘sudo’ group so that you can issue the occasinal root or system administration command.

adduser yourChosenRegularUserName
usermod -aG sudo yourChosenRegularUserName

Add your PUBLIC ssh key to the regular user account

On your existing local workstation issue the ssh-copy-id command to upload your ssh key to the ~./ssh/authorized_keys file on the newly installed Debian system.

ssh-copy-id -i ~/.ssh/id_ed25519.pub yourChosenRegularUserName@new-system-ip-address

After issuing that command CHECK that you are able to login from your workstation to the new system.

ssh yourChosenRegularUserName@new-system-ip-address

Don’t have an SSH key on your workstation? Just create one. Issue the following command and follow the prompts.

ssh-keygen -t ed25519

TIP: Use a specific key for each system you want to reach. TIP: Tired of typing your password? When you generate your key leave the password fields blank.

Disable Password Authentication on the new system

Since you are now able to access the new system using your SSH key – instead of using a username/password combination – disable BOTH password authentication and root logins on the new system.

ssh yourChosenRegularUserName@new-system-ip-address
sudo nano /etc/ssh/sshd_config

Update or add the following entries to the sshd_config file:

PermitRootLogin no
PasswordAuthentication no

Save the file and restart the sshd server.

sudo systemctl restart sshd

Install and enable a firewall

The follwing commands will allow you to install a firewall AND also allow the new system to accept ssh connections on port 22.

sudo apt install ufw
sudo ufw status
sudo ufw allow 22
sudo ufw enable

TIP: Want to see what services are waiting for connections on the new system? Is the the ss comand:

sudo ss -tupln

Avoid confusion – give the new system a name

Giving the system an easy to understand name will help you avoid confusion.

TIP: Add the name of the system (and it’s IP address) to your DNS server.

Often the name of the system is set at install. Still, you can update the name of the system after install.

sudo vi /etc/hostname

Update the name of your new system. Example:

If the name in the hostname file is ‘localhost’ update the name to ‘yourChosenHostname’ except all lowercase letters.

Once you know what the IP address of your new system will be share this new name with your system administrator or if you are able update your DNS settings on your own. Optionally, add this information to the /etc/hosts file on the new system

sudo vi /etc/hosts

Example where 123.456.789.123 is the new-system-ip-address:

xxx.yyy.zzz.123 yourChosenHostname.your-domain.com yourChosenHostname

Save the file.

Addittional Resources

Quick Install of WildFly 14 for Development

This is a quick revisit of a topic I covered earlier at this site with a couple up updates to address WildFly 14.

https://dennis.gesker.com/2016/02/09/wildfly-10-on-ubuntu-15-10/

I just started playing with WildFly 14 TODAY. The WildFly project is well into the process of incorporating Java/Jakarta EE features. WildFly 14 is EE 8 certified.

Below is a quick recipe for getting WildFly onto your development machine quickly. So, here we go.

Open a terminal, get WildFly and fire it up…

$ mkdir ~/bin
$ cd ~/bin
$ wget --tries=0 --continue http://download.jboss.org/wildfly/14.0.0.Final/wildfly-14.0.0.Final.tar.gz
$ tar -xvzf wildfly-14.0.0.Final.tar.gz
$ ln -s wildfly-14.0.0.Final wildfly
$ ~/bin/wildfly/bin/standalone.sh -c standalone-full.xml

Leave WildFly running and fire up a new terminal window…

$ cd ~/bin/wildfly/bin
$ ./add-user.sh -u wildfly -p wildfly -r ManagementRealm

(Please use a WAY better username and password even in development.)

At this point WildFly is running, you have established a management user and can connect to the web console at http://127.0.0.1:9990/

I use PostgreSQL most often so let’s add a driver for that. It will also be handy to have this driver installed later If I need to add a jdbc realm for authentication and of course PostgreSQL data sources for the WildFly application server to use in conjunction with your application.

$ cd /tmp
$ wget --tries=0 --continue https://jdbc.postgresql.org/download/postgresql-42.2.5.jar
$ cd ~/bin/wildfly/bin
$ ./jboss-cli.sh --connect
[ standalone@127.0.0.1:9990 ] module add --name=org.postgresql --resources=/tmp/postgresql-42.2.5.jar --dependencies=javax.api,javax.transaction.api
[ standalone@127.0.0.1:9990 ] /subsystem=datasources/jdbc-driver=postgres:add(driver-name="postgres",driver-module-name="org.postgresql",driver-class-name=org.postgresql.Driver)
[ standalone@127.0.0.1:9990 ] \q
Note:
When you are ready to setup a datasource later on using the posgresql jdbc driver the command you issue will be similar to:
[ standalone@127.0.0.1:9990 ] data-source add --jndi-name=java:jboss/datasources/YourSourceDS --name=YourDataSourceDS --connection-url=jdbc:postgresql://localhost:5432/yourdatabasename --driver-name=postgres --user-name=yourPostgresqlUsername --password=yourPostgresqlPassword

 

Also, most of my projects have a GIS component or require some GIS functionality so let’s add that while we are setting up WildFly in our dev environment. BTW, with a little luck this next step will be easier in the future. I’ve put in a feature request with the folks on the WildFly team who work closely with the folks on the Hibernate ORM team.

These requests are at:

https://issues.jboss.org/browse/WFLY-7524
https://hibernate.atlassian.net/browse/HHH-12163

In the mean time switch back to the terminal where WildFly is running and enter Ctrl-C to stop the server. We’ll add the spatial component to Hibernate in WildFly manually.

$ cd ~/bin/wildfly/modules/system/layers/base/org/hibernate/main
$ mvn dependency:copy -Dartifact=org.hibernate:hibernate-spatial:5.3.6.Final:jar -DoutputDirectory=.
$ mvn dependency:copy -Dartifact=org.geolatte:geolatte-geom:1.3.0:jar -DoutputDirectory=.
$ mvn dependency:copy -Dartifact=org.geolatte:geolatte-geojson:1.3.0:jar -DoutputDirectory=.

Now, edit the module.xml file found in this same directory (~/bin/wildfly/modules/system/layers/base/org/hibernate/main)

Add the following snippet to the bottom of the <resources> section of the file:

<resource-root path="hibernate-spatial-5.3.6.Final.jar"/>
<resource-root path="geolatte-geom-1.3.0.jar"/>
<resource-root path="geolatte-geojson-1.3.0.jar"/>

Add the following snippet to the <dependencies> section of the file:

<module name="org.slf4j"/>
<module name="org.postgresql"/>

Finally, fire WildFly back up…

$ cd ~/bin
$ ~/bin/wildfly/bin/standalone.sh -c standalone-full.xml

At this point you should be all set to use the new EE 8 stuff on your development machine.
Below are a few links to pages on my own blog you may find helpful as you get to know WildFly or see warnings. They reference older versions of WildFly but should still be useful:
https://dennis.gesker.com/2016/02/16/wildfly-10-and-wflytx0013/
https://dennis.gesker.com/2016/02/16/add-gmail-mail-session-to-wildfly-10/
https://dennis.gesker.com/2017/08/02/wildfly-10-1-warnings-tweaking-ubuntu-17-04/
https://dennis.gesker.com/2016/02/16/wildfly-10-java-lang-outofmemoryerror-metadata-space/

Of, course hit https://WildFly.org for other specific tips and tricks.

And, I recommend joining the mailing lists for WildFly. The folks on this project seem really nice and I pick up a lot of tips and tricks just following along on various threads.

Watching the commits to the project on GitHub is also a plus for anyone new to WildFly as you can watch the project evolve in real time.

Good luck with your project!

Quick Install of WildFly 13 for Development

This is a quick revisit of a topic I covered earlier at this site with a couple up updates to address WildfFly 13.

https://dennis.gesker.com/2016/02/09/wildfly-10-on-ubuntu-15-10/

I just started playing with WildFly 13. The WildFly project is well into the process of incorporating Java/Jakarta EE features. I believe at this time that WildFly will be EE 8 compliant when the project releases WildFly 14. But, I’d like to play with these features now so it is really nice that WildFly 13 has a startup mode to let one use EE 8.

Roadmap information on any application server is best read directly from the project so check out the release page for WildFly 13 from May:

http://wildfly.org/news/2018/05/30/WildFly13-Final-Released/
Below is a quick recipe for getting WildFly onto your development machine quickly. So, here we go.

Open a terminal, get WildFly and fire it up…

$ mkdir ~/bin
$ cd ~/bin
$ wget --tries=0 --continue http://download.jboss.org/wildfly/13.0.0.Final/wildfly-13.0.0.Final.tar.gz
$ tar -xvzf wildfly-13.0.0.Final.tar.gz
$ ln -s wildfly-13.0.0.Final wildfly
$ ~/bin/wildfly/bin/standalone.sh -c standalone-ee8.xml

(The standalone-ee8.xml configuration fire turns on all the new EE 8 stuff)

Leave WildFly running and fire up a new terminal window…

$ cd ~/bin/wildfly/bin
$ ./add-user.sh -u wildfly -p wildfly -r ManagementRealm

(Please use a WAY better username and password even in development.)

At this point WildFly is running, you have established a management user and can connect to the web console at http://127.0.0.1:9990/

I use PostgreSQL most often so let’s add a driver for that. It will also be handy to have this driver installed later If I need to add a jdbc realm for authentication and of course PostgreSQL data sources for the WildFly application server to use in conjunction with your application.

$ cd /tmp
$ wget --tries=0 --continue https://jdbc.postgresql.org/download/postgresql-42.2.4.jar
$ cd ~/bin/wildfly/bin
$ ./jboss-cli.sh --connect
[ standalone@127.0.0.1:9990 ] module add --name=org.postgresql --resources=/tmp/postgresql-42.2.4.jar --dependencies=javax.api,javax.transaction.api
[ standalone@127.0.0.1:9990 ] /subsystem=datasources/jdbc-driver=postgres:add(driver-name="postgres",driver-module-name="org.postgresql",driver-class-name=org.postgresql.Driver)
[ standalone@127.0.0.1:9990 ] \q
Note:
When you are ready to setup a datasource later on using the posgresql jdbc driver the command you issue will be similar to:
[ standalone@127.0.0.1:9990 ] data-source add --jndi-name=java:jboss/datasources/YourSourceDS --name=YourDataSourceDS --connection-url=jdbc:postgresql://localhost:5432/yourdatabasename --driver-name=postgres --user-name=yourPostgresqlUsername --password=yourPostgresqlPassword

Also, most of my projects have a GIS component or require some GIS functionality so let’s add that why we are setting up WildFly in our dev environment. BTW, with a little luck this next step will be easier in the future. I’ve put in a feature request with the folks on the WildFly team who work closely with the folks on the Hibernate ORM team.

These requests are at:

https://issues.jboss.org/browse/WFLY-7524
https://hibernate.atlassian.net/browse/HHH-12163
In the mean time switch back to the terminal where WildFly is running and enter Ctrl-C to stop the server. We’ll add the spatial component to Hibernate in WildFly manually.

$ cd ~/bin/wildfly/modules/system/layers/base/org/hibernate/main
$ mvn dependency:copy -Dartifact=org.hibernate:hibernate-spatial:5.3.1.Final:jar -DoutputDirectory=.
$ mvn dependency:copy -Dartifact=org.geolatte:geolatte-geom:1.3.0:jar -DoutputDirectory=.
$ mvn dependency:copy -Dartifact=org.geolatte:geolatte-geojson:1.3.0:jar -DoutputDirectory=.
$ mvn dependency:copy -Dartifact=org.hibernate:hibernate-spatial:5.1.14.Final:jar -DoutputDirectory=.
$ mvn dependency:copy -Dartifact=org.geolatte:geolatte-geom:1.0.6:jar -DoutputDirectory=.
$ mvn dependency:copy -Dartifact=org.geolatte:geolatte-geojson:1.0.6:jar -DoutputDirectory=.

Now, edit the module.xml file found in this same directory (~/bin/wildfly/modules/system/layers/base/org/hibernate/main)

Add the following snippet to the bottom of the <resources> section of the file:

<resource-root path="hibernate-spatial-5.3.1.Final.jar"> <!-- ${version.org.hibernate53} -->
<conditions>
<property-equal name="ee8.preview.mode" value="true"/>
</conditions>
</resource-root>
<resource-root path="hibernate-spatial-5.1.14.Final.jar">
<conditions>
<property-not-equal name="ee8.preview.mode" value="true"/>
</conditions>
</resource-root>

<resource-root path="geolatte-geom-1.3.0.jar">
<conditions>
<property-equal name="ee8.preview.mode" value="true"/>
</conditions>
</resource-root>
<resource-root path="geolatte-geojson-1.3.0.jar"> 
<conditions> <property-equal name="ee8.preview.mode" value="true"/> 
</conditions> 
</resource-root>
<resource-root path="geolatte-geom-1.0.6.jar">
<conditions>
<property-not-equal name="ee8.preview.mode" value="true"/>
</conditions>
</resource-root>
 <resource-root path="geolatte-geojson-1.0.6.jar">
<conditions>
<property-not-equal name="ee8.preview.mode" value="true"/>
</conditions>
</resource-root> 

Add the following snippet to the <dependencies> section of the file:

<module name="org.slf4j"/>
<module name="org.postgresql"/>

Finally, fire WildfFly back up…

$ cd ~/bin
$ ~/bin/wildfly/bin/standalone.sh -c standalone-ee8.xml

At this point you should be all set to use the new EE 8 stuff on your development machine.
Below are a few links to pages on my own blog you may find helpful as you get to know WildFly or see warnings. They reference older versions of WildFly but should still be useful:
https://dennis.gesker.com/2016/02/16/wildfly-10-and-wflytx0013/
https://dennis.gesker.com/2016/02/16/add-gmail-mail-session-to-wildfly-10/
https://dennis.gesker.com/2017/08/02/wildfly-10-1-warnings-tweaking-ubuntu-17-04/
https://dennis.gesker.com/2016/02/16/wildfly-10-java-lang-outofmemoryerror-metadata-space/
Of, course hit https://WildFly.org for other specific tips and tricks.

And, I recommend joining the mailing lists for WildFly. The folks on this project seem really nice and I pick up a lot of tips and tricks just following along on various threads. Watching the commits to the project on GitHub is also a plus for anyone new to WildFly as you can watch the project evolve in real time.

Good luck with your project!

Blockchain in a Nutshell in English

Blockchain in a Nutshell in English
by Dennis Gesker

Those of us that work with data — or put another way — all of us that use computers implicitly understand that there are generally only four things that one can do with data. We can create new data, retrieve or recall data, update existing data, and of course delete data. These four tasks create a convention of understanding how we generally interact with data that even has a fun descriptive name “the CRUD cycle”; Create, Retrieve, Update and Delete.

So much of what we regularly do with computers and computer systems of all types — PCs, phones, databases, spreadsheets, etc. — that we hardly even think about the process of manipulating data. One might go even far as to say we rarely consider this convention except when some pesky software or system failure obstructs one of these actions as in the case where perhaps a website won’t process an order or even worse when a system we use accelerates one of these actions in a way we don’t find helpful like accidentally deleting all our photos.

However, a new convention with regard to the life cycle of data is gaining in popularity. Although there are many implementations of this new data life cycle the convention has caught popular attention in that it underlies exciting and novel technologies like cryptocurrencies, digital money, the most well know being Bitcoin; digital money. This new data convention underlying Bitcoin and similar non-government backed currencies and other applications under development is gaining popularity is called “Blockchain.”

In some respects blockchain technology can be thought of as a degraded form of the CRUD cycle. Unlike the CRUD cycle in blockchain one cannot (generally speaking) UPDATE or DELETE data. One can only retrieve data and append data.

And, even these two functions have a spin on them. When one appends data it is as if a new link is added to a chain. And, the new link or record (record block) is both linked and secured using timestamps and cryptographic techniques to protect the contents and assure others that the new record has been successfully added without tampering. And there is very likely many copies of the new record spread about the internet so a user — with the correct access ability — might retrieve or recall the record from a location other than where it was originally written. So, blockchain is said to be resistant to updates or modification and somewhat more durable because it is often implemented in a distributed fashion (copies are spread around on different systems) and the cryptographic mechanisms prevent or at least warn of record tampering attempts.

So, it behaves much like old paper and pen ledger books in that once an entry is made — even if the entry is wrong — the entry can’t be undone. To correct the “bad entry” one would need to append or add a new entry making an adjustment to the earlier entry. But, because the earlier entry never comes out of the “chain” there is a history of how a set of transactions reached its current state.

Have you ever had the experience of working or a document with a colleague and being unable to recall who made and edit and when? This doesn’t happen in “blockchain” because old activity is always retained. The chain of events and data appended only grows.

Presently, digital or computer currencies seem to get most of the attention with regard to this underlying data and record convention but it isn’t a difficult jump of imagination to see that if this convention can work for purchases that it may lend itself to other business processes like contracts (what was in the contract document and when and who signed it) where tracking of revisions, acknowledgment of amendments and the date/time/signatures of the parties involved need to be tracked and verified.

There are many organizations putting new spins on this technology — faster processing, ease of use, side chains or channels for side deals, varying levels of who can see inside a record or transactions, as in the case where a third party may need to observe or intercede in a transaction or contract, and, I’m sure other novel uses.

This is definitely a technology to keep one’s eye on as it is inherently interesting but really the items to look for are regular everyday activities that it can ensure happen consistently and places where it is able to drive out cost. Blockchain absolutely has the potential to be “disruptive” to how we use and interact with computer systems and each other.

WildFly 10.1 Warnings & Tweaking Ubuntu 17.04

On a fresh workstation install of WildFly it may throws some warnings about memory and buffer resources available. A few tweak to these settings in Linux will resolve these warnings.

sysctl -w net.core.rmem_max=26214400
sysctl -w net.core.wmem_max=26214400
sysctl -w net.core.rmem_default=65536
sysctl -w net.core.wmem_default=65536
sysctl -w net.ipv4.tcp_rmem='4096 87380 8388608'
sysctl -w net.ipv4.tcp_wmem='4096 65536 8388608'
sysctl -w net.ipv4.tcp_mem='8388608 8388608 8388608'

 

If you wish for these settings to survive a reboot edit /etc/sysctl.conf and append the following lines to the bottom of the file:

net.core.rmem_max=26214400
net.core.wmem_max=26214400
net.core.rmem_default=65536
net.core.wmem_default=65536
net.ipv4.tcp_rmem='4096 87380 8388608'
net.ipv4.tcp_wmem='4096 65536 8388608'
net.ipv4.tcp_mem='8388608 8388608 8388608'
net.ipv4.route.flush=1

 

Quickly Reestablish Trust — Older Versions of Windows

Every once in a while an older workstation or server will lose its trust relationship with the Windows Domain. I experienced this “lost of trust” issue recently when I had to roll back a virtual server instance of  Windows Server 2008r2 to a snapshot that was quite dated.

The error message was similar to: “The trust relationship between this workstation and the primary domain failed.”

A quick Google search seems to turn up the popular solution of re-joining the domain which requires rebooting the server/workstation. This does work.

However, if you are in a rush (and you are able to log into the server/workstation with a local account) give the following command a try using the command prompt:

netdom.exe resetpwd /s:DomainServer /ud:YourDomain\Administrator /pd:*

 

Of course, substitute “DomainServer” with one of your actual domain servers and substitute “YourDomain” with the name of your domain.

“netdom.exe” is probably already installed if you need to try this little trick on an older version of Windows Server. On older versions of Windows (workstation) OS, for instance version 7, you may have to install the Remote Server Administration Tools which you can find at this link in order to get this utility installed.

 

Tor Proxy & Bridge on Ubuntu 16.10

The tor project is a collection of software designed as an Internet layer/protocol that enables users send/receive network traffic of all kinds while making it difficult for a third party observer to  conduct surveillance of the traffic. The project attempts to accomplish this by bouncing the user’s network traffic around a world wide network of relays. All of this traffic bouncing make the traffic difficult to observe and difficult for the computer systems your attempt to reach by way of the tor network to know your true physical location or ip address.

The network is operated by numerous volunteers who are willing to share some of their bandwidth and a little CPU power with the rest of the network. Most participants in the network feel that this technology is an extremely important component in the mix of available technologies that can help users maintain their privacy.

As with all technology the tor network can be used for the greater good or for less noble purposes. Examples of this usage dichotomy might include helping those who live in dangerous part of the world or under the rule of oppressive regimes communicate with the rest of the world or enabling end users to attain the intellectual property of others illegally. But, I suspect most users just want to maintain their privacy.

Below is a recipe you can use to install the tor software on your computer thereby creating a node or extra hop in the network to allow or assist in the bouncing of data around the tor network helping others maintain their privacy. The software is available for several platforms but we are going to focus on installing the software on an Ubuntu 16.10 Server.

Our configuration will allow our server to act as a proxy on the LAN so that other computers on your local area network can make use of the tor proxy we are setting up and hence have their traffic relayed and obfuscated over the tor network. This recipe will also allow our server to act as a “bridge” or specialized tor relay node that is “unpublished” and known (for discussion purposes)  by only a few other nodes on the network. Each bridge node increase the opacity of the network. So, even if your node isn’t a “exit” node it is a meaningful contribution to the operation of the network.

Installing the basic software…

We are going to elevate to root, add a few repository addresses at the tor project to ensure we are getting the latest and greatest versions of the software, we’ll update our local db of available software and install the required packages.

$ sudo -s
# vi (or your favorite editor) /etc/apt/sources.list.d/tor.list

Add the lines:

deb http://deb.torproject.org/torproject.org xenial main
deb-src http://deb.torproject.org/torproject.org xenial main
deb http://deb.torproject.org/torproject.org obfs4proxy main

Note: We are installing on Ubuntu 16.10 “Yakkety” but two of the entries above are for Ubuntu 16.04 “Xenial.” It’s OK. It seems that the required packages on the tor Xenial repository site are more up to date than those found in the official Ubuntu 16.04 Xenial repositories and they run just fine on Ubuntu 16.10 Yakkety release. I don’t know if the tor project will have a specific Ubuntu 16.10 Yakkety repository but for now the entries above will get the job done. This of course also begs the question why aren’t the most current stable versions of the tor packages already in the main Ubuntu repository? I don’t know the reason for this either. I suspect it’s probably a timing isssue of the when the software is added to the “upstream” Debian project from which Ubuntu draws many software packages. But, I am unsure if this supposition is correct.

Next we need to download and add the gpg key for the tor project repository so that we can be sure our standard apt tools to read from the repository and ensure that the packages are indeed those published by the project.

# gpg –keyserver keys.gnupg.net –recv A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89
# gpg –export A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89 | sudo apt-key add –

Now let’s update and install the software:

# apt-get install aptitude (my preferred apt tool)
# aptitude update
# aptitude install –with-recommends tor deb.torproject.org-keyring obfs4proxy tor-arm

Before we go any further there is a bit of a glitch/typo in one of the apparmor (firewall) files that is a part of the Ubuntu 16.10 distribution. We need to edit this file and fix the glitch/typo before we continue with the configuration of tor on our server.

# vi /etc/apparmor.d/abstractions/tor

Change 2 entries/lines in this file from:

/usr/bin/obfsproxy PUx,
/usr/bin/obfs4proxy PUx,

to:

 /usr/bin/obfsproxy ix,
/usr/bin/obfs4proxy ix,

After editing and saving the updated file issue the command:

# apparmor_parser -r /etc/apparmor.d/system_tor

Now that we’ve addressed the apparmor glitch/typo let’s start configuring tor. There are two files we need to address. In the first file of interest is the “default” file used by the tor init.d script. We just want to make a quick tweak and this step is optional.

# vi /etc/default/tor

Uncomment and alter the line:

# NICE=”–nicelevel 5″

to:

NICE=”–nicelevel 15″ (We removed the # and changed 5 to 15)

The impact of the above edit was to change the priority that the operating system will give to the tor proxy when it runs. 15 is actually a LOW priority so we know that tor won’t be intrusive on other items we have running on our server. Again, the above is optional and not critical.

Next, we’re going to clear out the default main configuration file for tor and just put in our own entries. But, we’ll back up the original file first just in case we need the file for reference later.

# cp /etc/tor/torrc /etc/tor/torrc.orig (back this file up just in case)
# vi /etc/tor/torrc

Make the following entries in the file. Of course, change 196.168.100.100 to the address of your server. Change the SOCKSPolicy network/mask to match your network. Update the Address and Nickname values to meet your needs.

SOCKSPort 9050 # Default: Bind to localhost:9050 for local connections.
SOCKSPort 192.168.100.100:9050 # Bind to this address:port so available to the LAN
SOCKSPort 192.168.100.100:9100 # Bind to this address:port so available to the LAN

SOCKSPolicy accept 192.168.0.0/16 # Allow any device on the LAN to use the proxy
SOCKSPolicy accept6 FC00::/7 # Allow any IP6 device on the LAN to use the proxy
SOCKSPolicy reject * # Deny anything else

Log notice file /var/log/tor/notices.log # Log all notices from the proxy
#Log debug file /var/log/tor/debug.log # Uncomment this line only if you really need it

RunAsDaemon 0 # init.d script is already running tor as daemon
DataDirectory /var/lib/tor # Default value
ControlPort 9051 # Default value
HashedControlPassword 16:872860B76453A77D60CA2BB8C1A7042072093276A3D701AD684053EC4C # Probably should hash a new password and replace this default control password

HiddenServiceStatistics 1 # Default Value

ORPort 443 # Default value is actually 9001
#DirPort 80 # Commented out because we are configuring as a bridge
#DirPortFrontPage /etc/tor/tor-exit-notice.html # Commented out because we are configuring as a bridge
Address externalfacing.address.com # Change to suit
Nickname yourNodeNickName # Change to suit
OutboundBindAddress 192.168.100.100 # Change to suit
DisableDebuggerAttachment 0

AccountingMax 40 GB # Increase if you can be more generous per month (e.g 100 GB)
AccountingStart month 1 00:00 # Start counting the 40 GBytes above on 1st of month
RelayBandwidthRate 100 KB # Increase if you can be more generous (e.g. 2 MB)
RelayBandwidthBurst 200 KB # Increase if you can be more generous (e.g. 4 MB)

ContactInfo Your Name <you@whatever.com> # Just in case network operators need to reach your

ExitPolicy reject *:* # no exits allowed
BridgeRelay 1 # Yes, our node is to operate as a bridge
PublishServerDescriptor 0 # Don’t let the world know I’m here

ServerTransportPlugin obfs3,obfs4 exec /usr/bin/obfs4proxy managed # Obfuscate Traffic

All done setting configuring tor. Restart the tor service.

# service tor restart

Now, one interesting thing above. Tor usually runs on port 9001 but we are going to run it on port 443 (https) which is a very common port for secure Internet traffic and so is less likely to draw attention. Feel free to stick with port 9001 if you already have services or server on this system using port 443. Use of port 443 in this example is just a preference of mine.

A second interesting point is that we are using the obfs4proxy (which is also backwards compatible with more widely deployed version 3 obfs3proxy) which endeavors to make our traffic to/from our node less obvious so a third party observer.

Thirdly, we are setup as a “proxy” on your local area network. So, any PC or Program (web browser, irc, chat, etc.) that can make use of a proxy can use 192.168.100.100 (or really the actual address of your server on your LAN) on either port 9050 or port 9100.

Finally, it is important to know that all the above if for naught if you can’t connect to the tor network and the network can’t connect to your server. So you will need to make sure your Internet firewall/router is set to port forward incoming traffic on port 443 to your internal server on port 443 of the server that we just setup for proxy & bridge above.

You can monitor the activity of your node using the “arm” command which is kind of like top/htop for your tor node. And of course you can always see what is generally going on with your node by tailing the /var/log/tor/notices.log

# tail -f /var/log/tor/notices.log
# arm

To learn more about tor, alternative configuration options and other software from the tor project check out http://torproject.org.

KDE Neon: Now with Wayland!

This post may be a bit on the late side but I find the topic interesting enough to share with you. Recently, the Kde Neon project released a developer edition of their Linux distribution that included Wayland as the display server. Wayland purports to be a lighter, simpler and easier to use replacement for the ubiquitous X Window System and protocols.
I’ve taken this release for a spin. And, I must stress that it is a development release so it is not intended for production use so probably best to hold off on trying this on your main PC/Notebook. My first impression is this…

“Wow, I have nothing to report! It just worked.”

I’ve installed the distribution on an older System76 Gazelle notebook. This system has an Intel I7-3610Q 8 Core Processor Running 3.3 Ghz, 16 GB of RAM and so uses the 3rd Gen Core processor Graphics Controller. So, admittedly it’s a pretty “Plain Jane” system. Still, it’s been a super reliable system and I would have had no idea that it was running Wayland if I didn’t install it from this particular development release on my own.

Consider that an rock solid endorsement for KDE/Plasma/Wayland on this hardware configuration. When a technology works so well you don’t notice it that is a great thing. If Wayland indeed makes life easier for the end developers maintaining the window system, developers preparing drivers and of course those developers writing software for end users that will run atop the package all the better. And, just an impression with no metrics it “feels” snappy. Wayland does look to be an important evolution to the graphics subsystem and perhaps more distributions will adopt it as the default.

I’m looking forward to seeing what others experience with the transition from X to Wayland. I have come to understand there is still work to be done on the Wayland project with regard to drivers on higher end graphics cards and am excited to see how Wayland will evolve as these issues are addressed. I have two primary production workstations with pretty good graphics cards and I’d like to take as much advantage of the hardware as possible.


On another note though I’m pretty new to the Kde Neon distribution I like it. It is based on the most stable release of Ubuntu Linux but has a focus on using the most current KDE Plasma Desktop. I am a big fan of Ubuntu but was kind of curious as to why the KDE Neon project didn’t go “upstream” and just pivot directly off of the “notoriously and obsessively stable” (a good thing) Debian of which Ubuntu is a derivative. Again, not an issue just an item of curiosity. I should also mention KDE Neon ships as a pretty bare bones distribution.

I actually like this approach as I’m for anything that keeps the bloat in my system down. And, the packaging/repository system of Debian and its derivatives makes adding the packages that you use and enjoy a pretty simple matter. For me I just needed installed some basic productivity and development packages upon which I depend and a little bit of my own bloat if you will:


# apt-get install aptitude
# aptitude update
# aptitude dist-upgrade
# aptitude install --with-recommends kgpg libreoffice calligra gimp vim-gtk postgresql postgresql-contrib postgis qgis openjdk-8-jdk openjdk-8-demo openjdk-8-doc kdevelop kompare build-essential pkg-config libc6-dev m4 gcc g++ g++-multilib autoconf libtool libncurses5-dev zlib1g-dev automake git subversion ant maven k3b dragonplayer krfb chromium-browser kmahjongg fortune-mod cowsay screenfetch  boinc-manager  htop iotop smartmontools

There are still a few other packages I am using that don’t seem to be the standard repositories or are stale in the repositories. But, these can be easily downloaded from their respective project websites:

Smart-git– http://www.syntevo.com/smartgit/download?file=smartgit/smartgit-8_0_3.deb
Chrome — https://www.google.com/chrome/browser/desktop/index.html
Earth — http://www.google.com/earth/download/ge/agree.html

And, then there are a few packages that don’t seem to have debs/repositories and have to be downloaded and installed in a more manual fashion. These packages are widely used and it does seem odd that the most current stable versions these packages are not already in the “Universe” repositories:

Netbean — https://netbeans.org/downloads/
(Hopefully, this will change now that it is an Apache project.)

Intellij — https://www.jetbrains.com/idea/download/download-thanks.html?platform=linux
(C’Mon Jetbrains! At least add your own repository. You have a license/key activation system anyway.)

Gradle — https://services.gradle.org/distributions/gradle-3.1-all.zip
(Ironic as it is a build system and deb package creation/push could just be a task in the main project.)

WildFly — http://download.jboss.org/wildfly/10.1.0.Final/wildfly-10.1.0.Final.zip
(Maybe Ubuntu deb ignored because this project is tied closely to RedHat? Just a pity if that is the reason.)

And, of course there is the latest “toy” I’m playing with and find interesting — the new ZCash Crypto-Currency — that does have a deb/repository but is still a new project and as of yet requires a number of  additional configuration and setup steps. I’ve written these steps in a previous blog entry:

Zcash — https://gesker.wordpress.com/2016/11/04/zcash-on-ubuntu-debian/


In short, check out the KDE Neon distribution. I like it. You may find it suits you as well. And, if you have a spare system take the new KDE Neon Wayland development release for a spin. Hopefully, you will have as little to note with regard to KDE on Wayland as I have found. And, that is highest praise one can give to a subsystem!


Please post comments below and share your thoughts and experiences. More importantly if you come across bugs or regressions let the KDE Neon project know by filing a bug report.

ZCash on Ubuntu & Debian

ZCash is the newest crypto-currency on the block. As of this writing the project is at version 1.0.1 of its client. And, the project seems Debian/Ubuntu friendly as the project maintainers have been pushing the official client in pre-built deb packages for general consumption. Very nice. Thank you z.cash project!

There seems to be a lot of help/instructions on the web. I’ve tried to summarize what I have found at various locations into this recipe below. But, https://z.cash should be considered the final authority on setup and configuration.

This write up intends to provide quick and dirty instructions on:

  1. Installing the packages
  2. Configuring ZCash
  3. Controlling daemon and mining behavior with user level crontab entries

I make the following assumptions:

  1. You have the ability to elevate to root
  2. You are OK with running the ZCash node all the time
  3. You would like to mine coins in the background in your off hours
  4. You would like to participate in a mining pool
  5. You do not want to mine coins during your work hours
  6. You will be running the ZCashd daemon as a regular user
  7. You don’t want to bother with a init.d or systemd script for the moment

At the end of the recipe I pose a few questions. Please comment or add to the discussion if you wish. I hope in advance you find this write-up useful so let’s begin!

Elevate to Root and Install:

$ sudo -s
$ sudo apt-get install apt-transport-https
$ wget -qO – https://apt.z.cash/zcash.asc | sudo apt-key add –
$ echo “deb [arch=amd64] https://apt.z.cash/ jessie main” | sudo tee /etc/apt/sources.list.d/zcash.list
$ apt-get update && sudo apt-get install zcash

Become a regular user and complete the configuration:

Return to regular user after sudo tasks above…
$
exit

Prime ZCash (keys and zkSnark stuff)…
$ zcash-fetch-params

Create configuration directory…
$ mkdir -p ~/.zcash

Add entries to the configuration file…
$ echo “addnode=mainnet.z.cash” >~/.zcash/zcash.conf
$ echo “rpcuser=username” >>~/.zcash/zcash.conf
$ echo “rpcpassword=head -c 32 /dev/urandom | base64” >>~/.zcash/zcash.conf
$ echo ‘equihashsolver=tromp’ >> ~/.zcash/zcash.conf
$ echo ‘#gen=1’ >> ~/.zcash/zcash.conf
$ echo “#genproclimit=$(nproc)” >> ~/.zcash/zcash.conf

Note: the last two entries above are going into the zcash.conf file commented out. This is OK as we will set these parameters in the crontab entries below. You may decide you want to just run ZCash without my crontab entries so you’ll be able to just uncomment those configuration lines.

Create an address (You will use this instead of my address in the crontab entries below when mining coins and will add the suffix rig1, rig2, etc for each machine where you are mining coin.)…
$ zcash-cli z_getnewaddress
$ zcash-cli z_listaddresses

Make you user level crontab entries…
$ crontab -e

# Start the ZCash daemon on reboot using only one (1) core for mining
@reboot /usr/bin/zcashd -reindex -showmetrics=0 -gen=1 -genproclimit=1 -stratum=stratum+tcp://us1-zcash.flypool.org:3333 -user=t1PBheBAnP8iAPjCWbmPsrgyjW7mWpKv4Ct.rig1

# Kill the ZCash daemon running on one (1) core @ 5:25 PM — E.g After Work
25 17   * * *   /usr/bin/pkill zcashd
# Start the ZCash daemon using more processing power (8 cores instead of 1) — E.g After Work
30 17   * * *   /usr/bin/zcashd -reindex -showmetrics=0 -gen=1 -genproclimit=8 -stratum=stratum+tcp://us1-zcash.flypool.org:3333 -user=t1PBheBAnP8iAPjCWbmPsrgyjW7mWpKv4Ct.rig1

# Kill the ZCash daemon using many cores @ 7:25 AM — E.g. Before Work
25 07   * * *   /usr/bin/pkill zcashd
# Start the ZCash @ 7:30 AM using only one (1) core for mining — E.g. Before Work
30 07   * * *   /usr/bin/zcashd -reindex -showmetrics=0 -gen=1 -genproclimit=1 -stratum=stratum+tcp://us1-zcash.flypool.org:3333 -user=t1PBheBAnP8iAPjCWbmPsrgyjW7mWpKv4Ct.rig1

Note:  flypool.org is just the first mining pool I found that didn’t require an account so I have been using it for testing. Please leave a comment below if you know of a good one and wish to make a recommendation. In addition I have seen that there are other mining packages out there that can be compiled and installed with little trouble but for now I’m content to try out the base client for pool mining.

Note: genproclimit is just how many cores you wish to use when mining. At the command line issue the “nproc” command to learn/confirm how many cores you have in your system. Adjust the genproclimit in the crontab entries to suit your needs.

Note: Simply using zcashd -daemon isn’t a bad option either if your not worried about system responsiveness. Your crontab -e entry would just be @reboot /usr/bin/zcashd -daemon just make sure you uncomment the gen and genproclimit lines in ~/.zcash/zcash.conf

Items I’m curious about:

  1. What is the ETA on the complete/definitive manual?
  2. ETA on “official” Android/iOS app?
  3. Is it good practice to use a Tor proxy or is this overkill?
  4. Can all of the command line options found in “zcashd –help” be used in the ~/.zcash/zcash.conf file?
  5. Will there be “official” GUIs from the project to administer:
    1. mining (including pool)?,
    2. wallet(s)?
    3. transaction processing?
      1. If so I hope there it is implemented in KDE or Java SWING so that it behaves the same on Windows/Linux/Mac etc.
      2. Minimize to system tray on KDE Plasma/Gnome/Windows/Etc.
  6. Will there be an “official” z.cash mining pool hosted at z.cash?
    1. A mining contribution going back to the project might be nice.
  7. Will the daemon get some usability functionality? For instance…
    1. Mine when processor is available?
    2. Or, “nice” settings in the zcash.conf file?
    3. Or, more sophisticated date/time or sleep controls than afforded by my crontab approach above?
    4. Again, would be great if all this could be controlled and monitored via an “official” GUI
  8. Will the “official” client get GPU mining capabilities to take advantage of graphics cards?
    1. As mentioned above I have noticed some alternate mining software is becoming available but haven’t tried them as of yet.
  9. There are a number of “Crypto-Currency Exchanges” so will one or more of these be designated as “definitive” by the project?
    1. I’ve noticed that http://kraken.com has already added zcash trading
      1. Not a plug. Just an observation that it was fast considering the project has only been “live” for a short time.
  10. What will the peer review look like on the new currency?
  11. How much peer review does the project require to confirm that the currency is secure and anonymous?

Finally, thank you to the members of the Z.Cash project for the new Crypto-Currency! Please check them out directly at: https:/z.cash