Projects tester using VBox VM

Mon 2025/12/01 Maintenance project / management / virtualbox

Overview

• repo status: Repo Information
• installation: Common Setup

Summary

I have many repos that I need to maintain. Part of that is to double-check that it installs correctly and runs, unit tests and if possible verifies correctly.

To do that, I create a minimal Virtualbox VM and then create the repos one-by-one. I run the scripts in each one as appropriate and track any significant failures. see Repo Information for the current status.

Currently, I only run this VM for Ubuntu. I do run the same tests on macOS and MSYS2/Windows but less often.

Assumptions

I do have some assumptions that are needed by my development setup. That means you can use these scripts and instructions as starting point but you will have to modify for your individual development process.

• ~/.pypirc : if you publish to pypi.org
• ~/projects/cpip : sym link to NAS directory for holding various files
• userid group dialout: needed for USB/UART interactions
• add other SSH keys you need the VM to have access to
• add other tools/languages you need at the global level
• add other apps/tools you don't need and can be cleaned up
• any other global files, config files, etc. should be added to proj-tester

Run it - top level

• update proj-tester/pt-common for your userid, etc.
• double check the rest of the scripts and files in proj-tester for your system
• Follow instructions in [doc/host-to-vm-projtester.md] to create the VM and initialize the Ubuntu OS.
• Test all the projects you have in the VM

How to create a VM Project Tester

Steps on how to install and configure Ubuntu Server 24.04 running on a VirtualBox VM ("virtual machine").

• host: is the PC that runs VirtualBox (Vbox).
• guest: is the VM that is running in VBox.

Note I run VirtualBox (Vbox) on my host PC which is also Ubuntu 24.04. But it should work correctly on Windows or macOS.

Also see Ubuntu Desktop on VBox

Install the latest VBox:

sudo apt install virtualbox-7.2

preparation

Download the version you wish to install:

• Ubuntu 22.04 Desktop
• Ubuntu 24.04 Desktop

guest info

• choose a username for your guest account: e.g. guest_userid
• choose a password for your guest account: e.g. guest_password
• choose a name for the guest VM: guest_name (vm-02 in this doc)

ip addresses

• choose an IP address for your guest VM: e.g. 10.0.0.15
• know the IP address on your development PC: e.g. 10.0.0.xx
• know the hostname of your development PC: john26
• know the other hostnames listed in your /etc/hosts e.g. "john29" (my common NAS server)

shared directory

• a directory name to share e.g. "vm-shared"
• choose a directory on your host PC for the shared directory e.g. /home/host_userid/vm-shared
• choose a directory on the guest VM for the shared directory e.g. /home/guest_userid/vm-shared
• you can have different names, but it is much easier to remember the same name on both host and guest.

ethernet access

• I assumed that you may want to connect to the VM from multiple PC's on your local internet e.g. on a second PC.
• I also assumed you can assign a fixed IP address to the VM via your router e.g. 10.0.0.15

check virtualbox installation

On Host:

• remove previous versions of Vbox:
  • run dkpg -l | grep -i virtualbox
• Download Vbox for Ubuntu 24.04
  • TODO fill these out:
  • goto ??
  • download
  • run dkpg -i x.deb
• For using USB in VM:
  • Go to VirtualBox Downloads
  • Download the "VirtualBox Extension Pack" that matches your exact VirtualBox version.
  • double-click the .ext-pack file and install it
  • on Host (not guest!) sudo usermod -aG vboxusers $USER

create VM

• open vbox v7.2
  • click Machine
  • click New
    • name: guest_name (e.g. vm-02-proj-tester)
    • Folder: should default to your normal VM directory
    • ISO image: select the file .iso file you downloaded e.g. ~/Downloads/ubuntu-24.04.2-desktop-amd64.iso
    • Skip Unattended installation: select this
    • click Hardware
  • get Hardware
    • Base Memory: select 10240 MB
    • Processors: 1 CPU
    • Enable EFI: leave unchecked
    • click Hard Disk
  • get Hard disk
    • select Create a Virtual Hard Disk Now
    • Hard Disk File Location and Size: select 60 GB (minimum)
    • select VDI (VirtualBox Disk Image), leave Pre-allocate Full Size unchecked
    • click Finish
• Click settings
  • System:
    • deselect Floppy (select: optical and HD only)
  • Network:
    • Name enp89s0
    • Mac: 080027791603 (record the MAC address so that your VM will get the same IP address from your router)

initial Ubuntu installation

VBox VM is now created, but Guest Ubuntu is not installed

• select the VM, click Start
• put mouse in VM box
• Should show Grub boot terminal. "Try or Install Ubuntu" press Enter
• get Desktop screen
• click on bottom icon "Install Ubuntu"
• Choose your language: select English (or your language), click Next
• Accessibility in Ubuntu: select None (or any if you need them), click Next
• Select your keyboard layout: select English (US), click Next
• Connect to the internet: select "Use wired connection" or Wifi as appropriate
• Update available: click Update now
• Update ready: click close Installer
• click on icon (lower right of screen) "Install Ubuntu 24.04.3 LTS"
• repeat the above until: "How would you like to install Ubuntu?"
• select Interactive installation, click Next
• select Default Selection, click Next
• Install recommended proprietary software?
  • select Install third-party software
  • select download and install support for additional media formats
  • click Next
• How do you want to install Ubuntu
  • select erase disk and install Ubuntu
  • Advanced features: None Selected
  • click Next
• Create your account
  • Your name: guest_userid
  • Your computer's name: vm-02 (i.e. the hostname)
  • your username: guest_userid
  • require my password to log in: selected
  • click Next
• select your timezone: click Next
• Review our choices: click Install
• installation starts

initial setup of Ubuntu on VM

Installation process should take around 8 minutes

• get "...installed and ready to use", click "Restart Now"
• get "Please remove the installation medium"
  • click Devices
  • click Optical Drive
  • ensure .iso is not selected
  • click on desktop screen
  • press Enter
• system reboots; wait <1m...
• should see a login screen
• login with the userid and password you entered
• dlgbox "Welcome to ...", click Next
• "Ubuntu Pro", click Skip
• "Help Improve Ubuntu", select "No don't send", "click Next"
• "Get started ...", click "Finish"
• open settings, pin to dash
• unpin Help
• click "Appearance", choose a different Background
• click "Ubuntu Desktop", set icon size to 32
• Open Terminal, pin to dash
• enter stty sane

update settings for the VM

• power off VM; Use Ubuntu power off, not VM power off.
  • click on power switch icon
  • click on "Power Off..."
  • get dlgbox, click on "Power Off"
• in VBox Manager, right click on new machine
• click Settings
  • General: (double check)
    • Advanced:
      • shared clipboard: Bidirectional
      • Drag'n'Drop: Disabled
    • other defaults are ok
  • System: (double check)
    • deselect "Floppy" in "Boot Order"
    • other defaults are ok
  • Display: defaults are ok
  • Storage:
    • in "Controller: IDE", select "Empty"
    • click blue disk icon
    • select VBoxGuestAdditions_7.2.2.iso (or latest one)
    • defaults are ok
  • Audio: defaults are ok
  • Network
    • Adapter 1:
      • Attached to: Bridged Adapter
      • Name: enp89s0
      • MAC address: update to use the previous value (080027791603) used in your router, or accept the current one
      • Virtual Cable connected: selected
  • Serial Ports: defaults are ok
  • USB:
    • select USB 3.0
    • other defaults are ok
  • Shared Folders:
    • on host: open a terminal; mkdir -p ~/vm-shared
    • click Blue Folder+ icon
      • Folder Path:
        • right menu: select Other and then locate and select /home/$host_userid/vm-shared (note: this is on the host)
      • Folder Name: vm-shared
      • Mount-point: /home/$guest_userid/vm-shared (this is on the guest)
      • click Auto-mount
  • User Interface: defaults are ok
• click OK

Initial Ubuntu setup

• Start VM and login
• Note: if you get the update software dlgbox, close it, going to do that below.
• open a terminal

# guest:
ip address
# should be the expected IP address from your router e.g. 10.0.0.15

sudo apt install openssh-server -y
sudo apt install openssh-client -y

SNAPSHOT 1 - basic Ubuntu without any additional packages

At this point, the following are working:

• basic Ubuntu
• enable openssh server/client
• Take a snapshot:
  • shutdown the VM window using Vbox shutdown
  • click Machine
  • click "Take+" icon
  • enter "base-os v1" for Snapshot Name
  • use the list above for the Description
  • click OK

Note: I use this to redo and recheck installation instructions

install basic components

• Start VM and login
• go to host; send helper scripts to VM

# host:
cd ~/bin/proj-man
./do-pt-publish
# output should be a list of pt-xx files

# if you get ssh errors:
ssh-keygen -f '/home/guest_userid/.ssh/known_hosts' -R '10.0.0.15'
# try again

• open a terminal

# guest:
# see proj-tester/pt-common for the BIN_DIR directory
ls ~/pt-bin
# should be a full set of pt-* scripts

nano .bashrc
# add this line:
export PATH=$PATH:$HOME/pt-bin

• reopen the terminal or source .bashrc to pick up the change

# test:
pt-report
# should see something like:
   OK  : on VM: 'vm-02'
-------: VM_HOSTNAME   = vm-02
-------: VM_USERID     = guest_userid
-------: HOST_HOSTNAME = john26
-------: BIN_DIR       = /home/guest_userid/pt-bin
    inet 10.0.0.15/24 brd 10.0.0.255 scope global noprefixroute enp0s3
   OK  : rc=0: 
PING google.com (142.250.73.110) 56(84) bytes of data.
64 bytes from pnseaa-an-in-f14.1e100.net (142.250.73.110): icmp_seq=1 ttl=118 time=20.1 ms
64 bytes from pnseaa-an-in-f14.1e100.net (142.250.73.110): icmp_seq=2 ttl=118 time=19.4 ms

--- google.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 19.360/19.740/20.121/0.380 ms
   OK  : rc=0: 

• if it failed to connect to ethernet (could not ping google.com):
  • click Devices | Network | Connect Network Adapter
  • repeat: click Devices | Network | Connect Network Adapter
• add the reported IP address to the router DHCP list
• check Devices | Optical Drive | VBoxGuestAdditions is NOT selected
• click on "Remove Disk from Virtual Drive" to remove it
• run guest additions step 1

# guest
pt-guest-additions1
# will automatically powerdown

• restart the VM, login
• start a new terminal
• check Devices | Optical Drive | VBoxGuestAdditions is selected
• click on "VBoxGuestAdditions_7.2.2.iso" to add it
• run guest additions step 2

# guest
ls /media/guest_userid
# check there is only one(!) VBox_GAs_7.2.2 file

pt-guest-additions2
# will automatically reboot

• login
• start a new terminal
• check cut-n-paste integration

# on Host:
# select this text
pt-report

# on Guest:
# paste it in the terminal

• if it doesn't work
  • re-install build-essential and virtual-guest-utils (in that order)
  • power off and power on the VM
• do additional setup

# add/confirm vboxsf is a group
# add/confirm bashrc has the PS1 line
pt-check-groups
# the PS1 line will be automatically added, so run it again to double-check
pt-check-groups

# check it worked
source .bashrc
# the prompt should go red; this indicates you are in the Guest VM

check vm-shared permissions

# on Guest
ls -al vm-shared
<snip>
drwxrwx---  1 root    vboxsf  4096 Nov  8 20:58 .
drwxr-x--- 17 guest_userid guest_userid 4096 Nov  8 21:17 ..
# there may be additional files here
# the key is the "vboxsf" group

check shared folder

# host:
echo from host > ~/vm-shared/host.txt
# host:
ls ~/vm-shared
# should see host.txt
cat ~/vm-shared/host.txt

# if fails, check Devices | Shared Folders | Shared Folder Settings
# ensure Auto-mount is selected; click OK
# then in the terminal "sudo mount -a"

# check the other direction
# host:
echo from host > ~/vm-shared/host.txt
# host:
ls ~/vm-shared/host.txt
# should see host.txt
cat ~/vm-shared/host.txt

check USB

• For using USB in VM:
  • ensure USB is up (e.g. Arduino) is running and USB cable is connected
  • restart Guest VM
    • click on Devices
    • click on USB
    • There should be a: "QinHeng" which matches:
      • on the Host lsusb: Bus 003 Device 014: ID 1a86:7523 QinHeng Electronics CH340 serial converter
    • select QinHeng

    lsusb
    # should see "QinHeng ..."
    
    # power off the USB/Arduino
    lsusb
    # should not see "QinHeng ..."
    
    # power on the USB/Arduino
    lsusb
    # still no "QinHeng"
    
    # select it in Devices | USB (as above)
    lsusb 
    # should see "QinHeng ..."
    

• remove the GuestAdditions Disk:
  • in VBox Manager, select the VM
  • click on Settings
  • scroll to Storage
  • select Controller: IDE with the GuestAdditions.iso in it
  • click on the blue disk icon
  • click on "Remove Disk from Virtual Drive"

SNAPSHOT 2 - has git, SSH access bitbucket etc.

At this point, the following are working:

• guest additions
• mouse integration
• shared folders
• USB working
• Take a snapshot:
  • click Machine
  • click "Take+" icon
  • enter "base-os v2" for Snapshot Name
  • use the list above for the Description
  • click OK

Install git, access to Bitbucket

• if needed, restart Guest VM and login
• start a terminal
• update /etc/host on guest and host

# on host:
sudo nano /etc/hosts
# check entry for vm-02
# 10.0.0.15  vm-02         # (wired)  VM for proj-tester
# select from "# === start" to "# === end"  

# on guest:
sudo nano /etc/hosts
# paste 
# NOTE: cut-n-paste in nano may get line numbers
# you may have to manually delete the line numbers in the pasted lines 

# comment out vm-02 entry 
# 10.0.0.15  vm-02         # (wired)  VM for proj-tester
# uncomment out the Host entry
# 10.0.0.xx  john26        # (wired)  dev PC

# test:
pt-check-hosts
# will ping google, host machine (john26) and common server (john29)

• set up git and bitbucket to be able to clone a project

# do once: set up ssh key
# see https://support.atlassian.com/bitbucket-cloud/docs/set-up-personal-ssh-keys-on-linux/
pt-create-bb
# cut-n-paste the key into your bitbucket account
# if this already exists; delete the old one

# test:
pt-check-bb
# should see: You can use git to connect to Bitbucket.

• set up SSH key pairs on Host/Guest

# on host:
cd ~/.ssh
ssh-keygen -t ed25519 -C 'vm-02@your_userid.com'
# create vm-02-ed25519
cat vm-02-ed25519.pub
# cut-n-paste this into vm-02/.ssh/authorized_keys

nano config
# add entry:
# OK - login to vm-02 (VM proj-tester) - .xx
Host vm-02
 HostName 10.0.0.15
 User guest_userid
 Port 22
 IdentityFile /home/guest_userid/.ssh/vm-02-ed25519
 IdentitiesOnly yes

cd "$HOME/bin/proj-man"
./proj-tester/host-send-keys 

# guest:
pt-check-ssh-keys
# should see
✅ OK  : on VM: 'vm-02'
✅ OK  : file exists : '/home/guest_userid/.ssh/config'
✅ OK  : file exists : '/home/guest_userid/.ssh/git_rsa'
# etc.

SNAPSHOT 3 - has git, SSH access bitbucket etc.

At this point, the following are working:

• /etc/hosts setup
• git
• bitbucket key pair
• other SSH pairs
• no proj-man
• Take a snapshot:
  • click Machine
  • click "Take+" icon
  • enter "base-os v3" for Snapshot Name
  • use the list above for the Description
  • click OK

Install proj-man, cpip, etc.

• restart Guest VM
• login
• start a terminal

pt-install-proj-man
# takes a few minutes

pt-check-bashrc
nano ~/.bashrc
# remove any PS1 lines from before; these are setup correctly by bashrc-ubu 
 
# close the terminal
# open a new one or do "source .bashrc"
pt-check-bashrc

• install other ubuntu packages

pt-install-ubu full
# takes a few minutes

# check it:
pt-install-ubu fast

• install latest doxygen

pt-install-doxygen
# currently should be 1.15.0

• setup fastnas and cpip shared directories

# cpip
sudo nano /etc/fstab
# //10.0.0.24/fastnas /home/guest_userid/fastnas  cifs credentials=/etc/nas_cred,users,rw,uid=1000,gid=1000,file_mode=0664,dir_mode=0775,vers=3.0,_netdev 0 0

pt-check-cpip
# should see
-------: check fastnas
drwxrwxr-x  2 guest_userid guest_userid    0 Nov 23 19:46 .
drwxr-x--- 20 guest_userid guest_userid 4096 Nov 29 19:46 ..
drwxrwxr-x  2 guest_userid guest_userid    0 Nov 14 13:29 cpip
<snip>
# and 
-------: check cpip
drwxrwxr-x 2 guest_userid guest_userid    0 Nov 14 13:29 .
drwxrwxr-x 2 guest_userid guest_userid    0 Nov 23 19:46 ..
-rw-rw-r-- 1 guest_userid guest_userid   10 Oct 29 11:48 abc.cpp
drwxrwxr-x 2 guest_userid guest_userid    0 Oct 29 11:47 cpip
-rw-rw-r-- 1 guest_userid guest_userid 1263 Nov 14 13:29 cpip.json
<snip>

check proj-man is installed correctly

• check basic zdo/ztestit Note: ztestit and zdo are utilities I've written to help maintain and test all of my projects. You will have to run the equivalent for your repos.

# note ./pt-install-proj-man does a do-publish, 
# so zdo and ztestit should already be in the PATH

cd ~
zdo cd proj-man
# check: new terminal created, should be in proj-man

# in the new terminal:
# check git
zdo check-branch
# should see
OK   check_branch: repo: branch matches: master
OK   check_branch: subm: branch matches: v1.0.7 (tools/xplat_utils)
zdo rc=0

ztestit  # check install
# check Repo Information is updated for proj-man

SNAPSHOT 4 - has proj-man, all basic zdo and ztestit functions

At this point:

• other ubu common packages
• proj-man is installed and is functional
• web-ver, cpip is set up
• basic zdo and ztestit are working
• Take a snapshot:
  • power off using Ubuntu power off
  • click Machine
  • click "Take+"
  • enter "base-os v4" for Snapshot Name
  • click OK

ztestit - create and run projects

• restart Guest VM
• login
• start a terminal
• zdo run all pwd; should list all repos (currently 80 repos); only 1 exists
• zdo first; goes to proj-man
• ./do-admin repo-info-runid
• ztestit
• zdo next; creates xplat-utils-ut ztestit ; do_install full is automatic
• zdo next; creates xplat-utils-rb-ut ztestit ; do_install full is automatic
• zdo next; creates falcon-logger
• zdo next; creates medver-test
• zdo next; creates pyalamake: ztestit ; do_install full is automatic
  • ensure USB for arduino is turned on
  • select Devices | USB | WinHeng (2)
• repeat until done

To increase VM size

I did have a scenario where I guessed the expected disk size for the VM smaller than was actually needed. I tried various things. The only thing that worked was deleting all snapshots. Once that was done, the extra space appeared in gparted and so I was able to expand the VM.

• power off VM using VM
• delete all snapshots(!)

cd /home/guest_userid/VirtualBox VMs/vm-02-proj-tester
VBoxManage showmediuminfo disk "/path/to/your/vm/MyVM.vdi"

VBoxManage modifymedium disk vm-02-proj-tester.vdi --resize 60000  # 60GB
VBoxManage showmediuminfo disk "/path/to/your/vm/MyVM.vdi"

• ensure all snapshots are deleted (!)
• start VM, open a terminal

df -h
# should be /dev/sda2; check it is the new size: 60G

sudo apt install gparted
sudo gparted
# note the state of /dev/sda2 

# right click on /dev/sda2
# click resize
# click on green checkmark 
# click Apply
# click close
df -h
# should show 60G (or near it)
sudo reboot

• open a terminal and check the free space df -h