Difference between revisions of "Project:LHS Graphs and Visualizations"

Latest revision as of 11:52, 5 January 2016

LHS Graphs and Visualizations
Members	Elliot
QR code

Overview

I have supplemented the Cacti graphs that we have for LHS bandwidth and power with metrics that provide insight to the growth of our community and organisation over time.

To chart the metrics various bits of data are exposed in a Cacti friendly way.

Metrics

Number of members

We may have to wait 12 months before it becomes interesting.

This data is stored in an Sqlite database on Turing. See the Schema. It can be queried like so:

 SELECT COUNT(id)
 FROM users
 WHERE subscribed = true;

Cacti runs on chomsky but the members database is on Turing. There is a PHP script on Turing to expose the member numbers and then a script on chomsky to pull this in with a HTTP request.

Code: https://github.com/londonhackspace/monitoring

URL: http://london.hackspace.org.uk/member_stats.php

IRC statistics

See project: ircensus

Wiki statistics

We get this using the MediaWiki API:

 http://wiki.hackspace.org.uk/w/api.php?action=query&meta=siteinfo&siprop=statistics&format=xml

It returns:

 <?xml version="1.0"?>
 <api>
   <query>
     <statistics
       pages="759"
       articles="215"
       views="229656"
       edits="7368"
       images="186"
       users="166"
       activeusers="22"
       admins="61"
       jobs="13"
     />
   </query>
 </api>

A Perl script generates the following output for cacti: pages:759 articles:215 views:229656 edits:7368 images:186 users:166 activeusers:22 admins:61 jobs:13

Code: https://github.com/londonhackspace/monitoring

Mailing list activity

My thanks to JamesG for his assistance and '78.86.160.161' for the original idea. We poll and scrape the groups page for members and and message count.

 http://groups.google.com/group/london-hack-space

We have a script that outputs: members:693 messages:3123

Code: https://github.com/londonhackspace/monitoring

Space occupancy

See project: spacensus

@@ Line 1: / Line 1: @@
 {{Project|members=[[User:Teabot|Elliot]]}}
-===Overview===
+==Overview==
-I'd like to supplement [http://hack.rs/cacti/graph_view.php?action=tree&tree_id=1 the Cacti graphs] that we have for LHS bandwidth and power with metrics that provide insight to the growth of our community and organisation over time.
+I have supplemented [http://hack.rs/cacti/graph_view.php?action=tree&tree_id=1 the Cacti graphs] that we have for LHS bandwidth and power with [http://hack.rs/cacti/graph_view.php?action=tree&tree_id=5 metrics] that provide insight to the growth of our community and organisation over time.
-Initially I'd like to chart the following:
+To chart the metrics various bits of data are exposed in a Cacti friendly way.
-* Number of members
-* Wiki activity
-Later I'd like to investigate:
+==Metrics==
-* Website visitors and/or page impressions
+===Number of members===
-* Mailing list activity
-* Space occupancy
-==Phase 1==
+We may have to wait 12 months before it becomes interesting.
-To chart the initial metrics various bits of data will need to be exposed in a Cacti friendly way. I need help in getting access to these data sources so that I can write the various data input methods.
-====Number of members====
 This data is stored in an Sqlite database on [[Turing]].  See the [https://github.com/londonhackspace/hackspace-foundation-sites/blob/master/etc/schema.sql Schema]. It can be queried like so:
@@ Line 22: / Line 16: @@
    WHERE subscribed = true;
-And for pending users:
+Cacti runs on chomsky but the members database is on Turing. There is a PHP script on Turing to expose the member numbers and then a script on chomsky to pull this in with a HTTP request.
-  SELECT COUNT(id)
+''Code:'' https://github.com/londonhackspace/monitoring
-  FROM users
-  WHERE subscribed = false;
-To import historic data into rrdtool we can look at the date of members first payments - I probably won't bother with this:
-  SELECT t1.timestamp, COUNT(t1.id)
+''URL:'' http://london.hackspace.org.uk/member_stats.php
-  FROM transactions t1
-  LEFT JOIN transactions AS t2 ON t1.user_id = t2.user_id
-  AND t1.timestamp > t2.timestamp
-  WHERE t2.timestamp IS NULL
-  GROUP BY t1.timestamp;
-This bash script will generate the following output for cacti: '''<tt>subscribed:136 pending:2</tt>'''
+===IRC statistics===
-  #!/bin/sh
+''See project: [[Project:Ircensus|ircensus]]''
-  SQLITE=/usr/bin/sqlite
-  DATABASE=<file.db>
-  SUBSCRIBED=`$SQLITE $DATABASE "select count(id) from users where subscribed = 'true';"`
-  if [ $? -ne 0 ]; then
-    echo "NaN"
-    exit 1
-  fi;
-  PENDING=`$SQLITE $DATABASE "select count(id) from users where subscribed = 'false';"`
-  if [ $? -ne 0 ]; then
-    echo "NaN"
-    exit 1
-  fi;
-  echo "subscribed:"$SUBSCRIBED" pending:"$PENDING
-====Wiki statistics====
+===Wiki statistics===
-* We can get this using the MediaWiki API:
+* We get this using the MediaWiki API:
    http://wiki.hackspace.org.uk/w/api.php?action=query&meta=siteinfo&siprop=statistics&format=xml
@@ Line 77: / Line 49: @@
    </api>
-Perl to generate the following output for cacti: '''<tt>pages:759 articles:215 views:229656 edits:7368 images:186 users:166 activeusers:22 admins:61 jobs:13</tt>'''
-  #!/usr/bin/perl
+A Perl script generates the following output for cacti: '''<tt>pages:759 articles:215 views:229656 edits:7368 images:186 users:166 activeusers:22 admins:61 jobs:13</tt>'''
-  use strict;
+''Code:'' https://github.com/londonhackspace/monitoring
-  use LWP::Simple;
-  use XML::Simple;
-  eval {
-    my $file = get("http://wiki.hackspace.org.uk/w/api.php?action=query&meta=siteinfo&siprop=statistics&format=xml");
-    my $xs1 = XML::Simple->new();
-    my $doc = $xs1->XMLin($file);
-    my $first = 1;
-    foreach my $key (keys (%{$doc->{query}->{statistics}})){
-      if ($first eq 1) {
-        $first = 0;
-      } else {
-        print " ";
-      }
-      print $key . ":" . $doc->{query}->{statistics}->{$key};
-    }
-;
-  } or do {
-    print "NaN";
-  }
-==Phase 2==
+===Mailing list activity===
-====Mailing list activity====
+* My thanks to JamesG for his assistance and '78.86.160.161' for the original idea. We poll and scrape the groups page for members and and message count.
-* There is no API for Google Groups. Perhaps poll the RSS feed and count unrecognized message ids or look at the date?
-   https://groups.google.com/group/london-hack-space/feed/rss_v2_0_msgs.xml?num=50
+   http://groups.google.com/group/london-hack-space
-* We can also calculate the size of the list (members) from the [http://groups.google.com/support/bin/answer.py?hl=en&answer=46398 mailing list download]
+We have a script that outputs: '''<tt>members:693 messages:3123</tt>'''
-====Website visitors and/or page impressions====
+''Code:'' https://github.com/londonhackspace/monitoring
-The main site and the Wiki use Google Analytics and this has an API. This [http://code.google.com/apis/analytics/docs/gdata/gdataCommonQueries.html documented method] looks promising:
-  https://www.google.com/analytics/feeds/data?metrics=ga%3Avisits%2Cga%3Apageviews&start-date=2010-11-29&end-date=2010-12-13&max-results=50
-==Phase 3==
+===Space occupancy===
+''See project: [[Project:Spacensus|spacensus]]''
-====Space occupancy====
+[[Category:Projects]]
-* We could use a directional IR occupancy counter. I think that we already have something like this in the LHS stores.
+[[Category:Infrastructure]]
+[[Category:Space_Infrastructure_Projects]]

Difference between revisions of "Project:LHS Graphs and Visualizations"

Latest revision as of 11:52, 5 January 2016

Contents

Overview

Metrics

Number of members

IRC statistics

Wiki statistics

Mailing list activity

Space occupancy

Navigation menu

Search