Hello.
I would like to share my experience on storing, processing and visualizing
time series data.
I am a physics graduate student from UIC (University of IL at Chicago).
The application.
http://131.193.191.37/~mnm/temperatures/menu.php
Let me explain what that is.
Our research group has a beam-time this week at ChemMatCARS Sector-15 at
APS.
We wanted to look at the channels A and B of our own lakeshore 340 and
thermal probe connected to Keithley 2000. Both connected to our laptop
via GPIB ? USB cable.
Using python with pyVisa we read those 3 variables and write them to
the MySQL database of the desktop that is located at UIC- that is the
link above. The connection to the UIC desktop is via the guest wirelesses
and SSH tunnel.
This web application is a stripped down version of something I am
currently developing for our in-house experiment at UIC that is being
controlled with EPICS.
In general there is a need for visualization of real time data in the
scientific community. However setting up and maintaining the
infrastructure is not a trivial matter. With this kind of centralized
system this problem could be significantly reduced.
For example for an EPICS environment such as the beamlines at APS the task
of visualizing small number of PV s could be reduced down to running a
simple python script on the client machine given that pyEpics installed on
the client machine. The database server has to be setup only once at one
location.
My rough estimate is that a single modern desktop system can serve tens
of clients simultaneously each individually recording at a rate of 10Hz.
The main requirement is fast hard drive array.
I have been thinking for a while starting a project on github to make this
web based application more general and would like to see if there is
anybody else doing something similar and would like to collaborate on
that.
One main distinctive feature of my database back-end is that I can process
tens of millions of records in as little as 2 milisecods and on average
less than 50ms which gave me solid ground for building all the dynamic
interactivity on the front end.
Miroslav
- Replies:
- RE: Processing And Visualization of time series data. Chen, Xihui
- Re: Processing And Visualization of time series data. Matt Newville
- Navigate by Date:
- Prev:
Re: Re: Is it possible to create a cross-compile environment using cygwin and vxWorks on Windows XP 송영기
- Next:
EPICS and MEDM on ubuntu nalaka samarasinghe
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
<2012>
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
- Navigate by Thread:
- Prev:
Re: Re: Is it possible to create a cross-compile environment using cygwin and vxWorks on Windows XP 송영기
- Next:
RE: Processing And Visualization of time series data. Chen, Xihui
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
<2012>
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
|