Overview
| PyPi module | https://pypi.org/project/on-the-fly-stats.html |
| git repository | https://bitbucket.org/arrizza-public/on-the-fly-stats |
| git command | git clone git@bitbucket.org:arrizza-public/on-the-fly-stats.git |
| Verification Report | https://arrizza.com/web-ver/python-on-the-fly-stats-report.html |
| Version Info |
|
- installation: see https://arrizza.com/setup-common
Summary
This Python module contains various statistic functions (e.g. standard deviation) that can be updated on the fly i.e. the entire dataset is not needed up front.
- See Quick Start for information on using scripts.
- See xplat-utils submodule for information on the submodule.
Sample code
see sample.py for a full example
from on_the_fly_stats import OTFStats
stats = OTFStats()
# create an instance of the std deviation
stats.create_stddev('some_tag')
# add some values to it
stats.update_stddev('some_tag', 1)
stats.update_stddev('some_tag', 2)
stats.update_stddev('some_tag', 3)
# have other instances of it
stats.update_stddev('other_tag', 0.1)
stats.update_stddev('other_tag', 0.2)
stats.update_stddev('other_tag', 0.3)
# show the current data for a specific instance
print(stats.stddev['some_tag'].stddev)
print(stats.stddev['some_tag'].variance)
print(stats.stddev['some_tag'].mean)
print(stats.stddev['some_tag'].num_elements)
# show the current data for a specific instance
print(stats.stddev['other_tag'].stddev)
# some a summary report
stats.set_report_writer(print)
stats.report_minimal()
# skip the headers for even less data
stats.report_minimal(headers=False)
Min/Max
Holds the minimum and maximum of the values provided so far
import random
from on_the_fly_stats import OTFStats
stats = OTFStats()
stats.create_min_max('minmax1')
for _ in range(10):
stats.update_min_max('minmax1', random.randint(0, 10))
print(stats.min_max['minmax1'].minimum)
print(stats.min_max['minmax1'].maximum)
print(stats.min_max['minmax1'].num_elements)
Average
Holds the average of the values provided so far
import random
from on_the_fly_stats import OTFStats
stats = OTFStats()
stats.create_average('avg1')
stats.update_average('avg1', random.uniform(0, 10))
stats.update_average('avg1', random.uniform(0, 10))
print(stats.average['avg1'].average)
print(stats.average['avg1'].anum_elementss)
Standard Deviation
Holds the standard deviation, mean and variance of the values provided so far
import random
from on_the_fly_stats import OTFStats
stats = OTFStats()
stats.create_stddev('sd1')
stats.update_stddev('sd1', random.random())
stats.update_stddev('sd1', random.random())
stats.update_stddev('sd1', random.random())
print(stats.stddev['sd1'].stddev)
print(stats.stddev['sd1'].mean)
print(stats.stddev['sd1'].variance)
print(stats.stddev['sd1'].num_elementss)
Counters
Holds counters of the values provided so far
from on_the_fly_stats import OTFStats
stats = OTFStats()
# automatically create a counter
stats.inc_counter('counter1')
stats.inc_counter('counter1') # should be 2
stats.dec_counter('counter2') # should be -1
stats.inc_counter('counter3')
stats.dec_counter('counter3') # should be 0
print(stats.counters['counter1'].count)
print(stats.counters['counter2'].count)
print(stats.counters['counter3'].count)
Reports
There are two builtin reports:
- a minimal report
- a full report
To use the builtin reports a function that writes a single line must be provided. The simplest is to use print().
stats.set_report_writer(print)
stats.report_minimal()
stats.report()
Minimal report output
The minimal report shows each stat kept and minimal data with minimal formatting
Min/Max:
0 7 minmax1
0.019647 0.825751 minmax2
Average:
4.495221 avg1
Stddev
111.109474 stddev1
0.300395 stddev2
Counters:
2 counter1
-1 counter2
0 counter3
Full report output
The full report has more formatting and layout
---- Stats:
Min Max statistic
--------------- --------------- ------------------------------------------------------------------
0 7 minmax1
0.019647 0.825751 minmax2
>>> end of min/max
Average statistic
--------------- ------------------------------------------------------------------
4.495221 avg1
>>> end of Averages
StdDev statistic
--------------- ------------------------------------------------------------------
111.109474 stddev1
0.300395 stddev2
>>> end of StdDev
Total statistic
--------------- ------------------------------------------------------------------
2 counter1
-1 counter2
0 counter3
>>> end of counters