CPIP commands and procedures

Overview

Used to create a C/C++ (and other techs) package manager, like PIP for python.

see CPIP Packages - the current packages available
these projects currently generate CPIP packages:
- see CPIP Common - C/C++ code for a logger
- see Find The Best - a tkinter GUI. The executables are published (tar/zip state)
- see C++ On the Fly Stats - C++ code for doing stats "on the fly"
see xplat_utils UTs - unit tests for the zpm functions that work with CPIP. see ut/test_tp025, tp026 and tp027 (so far)

How to create and publish a CPIP package

1. Gen files

Note: the examples below assume that the root cpip directory is in "~/projects/cpip". This can be changed in xplat.cfg, see xplat cfg

go to the repo that contains the file to publish
determine the files to publish
update tools/cpip_publish.json
- see below for the format of that json content
run ./do_build (if a python executable e.g. find-the-best)
- the cpip_publish.json will have a path to the executable in the dist/*, src/*, or lib/* files to publish
run “zpm cpip-publish”
- this takes the content of the cpip_publish.json and uses it to update the cpip.json and files in "~/projects/cpip"
check it on ubuntu “find ~/projects/cpip/msys2” (or whichever directory)
- ensure all the files were copied there
- ensure the content of ~/projects/cpip/cpip.json contains the new/updated package information
repeat this on each platform: macos, msys2 as needed

2. test the setup

I have a VM machine that replicates my website. I use this ("VM04") for test purposes to ensure that any changes I make are okay to finally publish to arrizza.com. VM04's local subnet ip address is 10.0.0.23.

go to ubuntu
start VM4
in my website repo, I have a special command “zpm doit push-cpip” that publishes to the website
go to https://arrizza.com/web-cpip and check it is there along with the previous ones

3. upload to your website

I use various tools and python scripts I've written to generate and update my website pages.

The key aspects are:

where should the file reside on the web server?
what page generation or support scripts are needed to for the CPIP package page?
which OS platforms are you supporting? What specific changes do they need?

How to use a CPIP package

download package from website using xplat.cfg

set the package(s) you want to use in xplat.cfg in section cpip.packages

[cpip]
root_dir = ~/projects/cpip
root_dir.win = y:/cpip
; used for local server testing of CPIP
;server_root = 'https://10.0.0.10/web-cpip'
; list of packages
packages = logger

Note: if packages is empty, then only the cpip.json file is updated.

zpm cpip-pull
---> cpip-pull: starting cpip: pull ...
     cpip-pull: pulling from: https://arrizza.com/web-cpip to /home/arrizza/projects/cpip
     cpip-pull: pkg: logger pulling...
        > cpip/logger/logger.h
        > cpip/logger/logger.cpp
---> cpip-pull: overall rc=0

go to the directory specified in cpip.root_dir (typically ~/projects/cpip)

$ ls ~/projects/cpip
cpip  cpip.json
$ ls ~/projects/cpip/cpip/logger
logger.cpp  logger.h

to manually download a single file

Goto CPIP Packages
click on the package you want to download
if it is not an "all" platforms package, make sure you select the OS you want to download for
Typically the browser will download to your ~/Downloads directory
open that directory on your PC

to manually download a compressed file

Note: some files may be compressed (e.g. executables) using tar or zip.

Goto CPIP Packages
click on the package you want to download
if it is not an "all" platforms package, make sure you select the OS you want to download for
Typically the browser will download to your ~/Downloads directory
Use either tar or zip to uncompress the file
Note: for some executables and on some platforms (e.g. Windows or macOS), you will not be able to run the executable without manually uncompressing the file

to get compilation information about a package

Use python to get information about C/C++ package.

The information:

include_dir: holds a path to any include directories
src : list of paths to each C/C++ source file (no includes)

from tools.xplat_utils.zpm_utils import ZpmUtils

pkgname = 'logger' 

utils = ZpmUtils()
pkginfo = utils.cpip_get(pkgname)
if pkginfo is None:
    svc.abort(f'could not find info for cpip package: {pkgname}')

svc.log.line(f'include_dir: {pkginfo.include_dir}'
for src_path in pkginfo.src:
    svc.log.line(f'src_path: {src_path}'

Content of tools/cpip_publish.json

This is a description of the content of the tools/cpip_publish.json file. It is used to describe a package for use in CPIP.

deleted

must be present
format: boolean, true/false
if true, the package has been deleted, ok to ignore on web page. If false the package will be published and will be available for download.

For example:

{
"abc": {
    "deleted": true,
    "desc": "ignore; for cpip testing only",
    "dst": ".",
    "dst-pattern": "{file}",
    "all.paths": { "lib/other/abc.cpp": {} }
}
}

desc

must be present
format: text

Short description of the package. This is used to generate the web-cpip page

dst

must be present
format: path

This is the path in ~/projects/cpip/* that the files/executeables will exist in. For the “all” platform, I typically use “cpip” and then files in all.src are relative to that directory. For other platforms, I typically use "." so the paths in "ubuntu.src", etc. are relative to that directory.

{
"logger": {
    "deleted": false,
    "desc": "c/c++ source files<br/> logging to stdout",
    "dst": "cpip",
    "dst-pattern": "logger/{file}",
    "all.paths": { "lib/logger/logger.h": {}, "lib/logger/logger.cpp": {} }
}
}

dst-pattern

must be present
format: path with symbols in it

Used to specify a pattern for the location in the ~/projects/cpip/* directory.

For example:

"dst": "cpip",
"dst-pattern": "logger/{file}"

will result with files put in ~/projects/cpip/cpip/logger/* directory.

all.paths

must be present if the same files are used across all platforms, otherwise must not be present (use "macos.paths" etc.)
format: dict of dictionaries
contains a set of file paths relative to the repo root directory

For example, there will be two files in the repo_dir/lib/logger/* directory json "all.paths": { "lib/logger/logger.h": {}, "lib/logger/logger.cpp": {} }

The {} indicates that no compression is needed for this file.

msys2.paths, macos.paths, ubuntu.paths

must be present if there are specific files used on specific OS platforms, otherwise must not be present
format: dict of dictionaries
Each key is a local path relative to the repo root directory.
If the element "compress" is specified:
- if it is "", it uses the default compression tool for that platform:
  - ubuntu: tar
  - macos: zip
  - msys2: zip
- if it is "tar", then the tar compression tool is used. Note: it is assumed that tar is installed on that platform.
- if it is "zip", then the zip compression tool is used. Note: it is assumed that zip is installed on that platform.

For example:

{
  "msys2.paths": { "dist/find-the-best.exe": { "compress": "" } },
  "macos.paths": { "dist/find-the-best": { "compress": "zip" } },
  "ubuntu.paths": { "dist/find-the-best": { "compress": "tar" } }
}

Content of cpip.json

This is a description of the content of the cpip.json file

deleted

must be present
format: boolean, true/false
if true, the package has been deleted, ok to ignore on page. If false the package is available for download.

desc

must be present
format: text
describes the package. This is used to generate the web-cpip page

src-proj

must be present
format: text
name of the repo name used to generate this package. for example, see bitbucket find-the-best

dst

must be present
format: path

all.src

must be present if the same files are used across all platforms, otherwise must not be present (use "macos.src" etc.)
format: list

contains a list of file paths

web site is relative to dst path: "web.../web-cpip/{dst}/{path in all.src}"
src paths are relative to ~/projects/cpip/{dst}/{path in all.src}

"cpp-on-the-fly": {
    "deleted": false,
    "desc": "c/c++ on-the-fly statistics",
    "src-proj": "cpp-on-the-fly-stats",
    "dst": "cpip",
    "all.src": ["cpp_otf/otf_list.h", "cpp_otf/otf_list.cpp", "cpp_otf/otf_stats.h", "cpp_otf/otf_stats.cpp"]
},

msys2.src

must be present if there are specific files used on Windows/MSYS2, otherwise must not be present
format: list

contains a list of file paths

web site is relative to dst path: "web.../web-cpip/msys2/{dst}/{path in msys2.src}"
src paths are relative to ~/projects/cpip/msys2/{dst}/{path in msys2.src}

"find-the-best": {
    "deleted": false,
    "desc": "find the best app to sort options by criteria",
    "src-proj": "find-the-best",
    "dst": ".",
    "msys2.src": ["find-the-best.zip"],
    "macos.src": ["find-the-best.zip"],
    "ubuntu.src": ["find-the-best.tar.gz"]
}

macos.src

must be present if there are specific files used on macOS, otherwise must not be present
format: list
contains a list of file paths
- web site is relative to dst path: "web.../web-cpip/macos/{dst}/{path in msys2.src}"
- src paths are relative to ~/projects/cpip/macos/{dst}/{path in msys2.src}

ubuntu.src

must be present if there are specific files used on ubuntu, otherwise must not be present
format: list
contains a list of file paths
- web site is relative to dst path: "web.../web-cpip/ubuntu/{dst}/{path in msys2.src}"
- src paths are relative to ~/projects/cpip/ubuntu/{dst}/{path in msys2.src}