# Visualizing-Streaming-Data **Repository Path**: bit2atom/Visualizing-Streaming-Data ## Basic Information - **Project Name**: Visualizing-Streaming-Data - **Description**: examples for a book by the same name - **Primary Language**: HTML - **License**: MIT - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2020-08-25 - **Last Updated**: 2020-12-19 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # Visualizing-Streaming-Data ![](cover.jpg) This is intended to be a companion to the book here:
https://www.amazon.com/Visualizing-Streaming-Data-Interactive-analysis/dp/1492031852/
Organized by chapter / subject.
if you have an issue getting something to work or want to suggest a new link / example please create an issue.
I'm also always looking for interesting places to apply some of these techniques. Let me know what your project is. ## Sources ### Links - https://aws.amazon.com/datasets/ - https://blockchain.info/api/api_websocket Blockchain streaming data - https://www.mediawiki.org/wiki/API:Recent_changes_stream Wikimedia network changes - https://www.honeynet.org/node/960 streaming honeypot events ### Code: [examples](./sources.md) ## Streaming ### Links - https://github.com/primus/primus mutliple client side library support - http://orientdb.com/docs/last/Live-Query.html streaming data feature - https://github.com/cayasso/mongo-oplog library to stream a mongodb oplog - http://papaparse.com/ stream a file - https://github.com/nevill/zongji stream a mysql binlog - https://github.com/SocketCluster/socketcluster distributed data streaming with channels - https://www.pubnub.com/ commercial service for streaming data - https://pusher.com/ commercial service for streaming data - https://www.elastic.co/products/beats the collection and streaming components for a very popular dstrbuted log system - https://aws.amazon.com/kinesis/ a cloud streaming framework by Amazon - https://kafka.apache.org/ streaming technology for very large scale implementations - https://osquery.io/ turn OS information into queryable data ## Processing ### Links - http://nytlabs.com/streamtools/ stream processing tools with workflow visualization - https://trackingjs.com/ image processing - http://rpominov.github.io/kefir/ stream processing library - http://reactivex.io/ stream processing library - https://momentjs.com/ transform between dat formats - http://numeraljs.com/ transform between number formats - https://github.com/SuddenDevelopment/json-collection-decorator JSON defined rules for processing ### Code: [examples](./processing.md) ## Client ### Links - - https://planetos.com/powerboard/ - https://www.alooma.com/blog/kafka-realtime-visualization a streaming data client for Kafka - http://ohm.ai - https://www.appdynamics.com/product/application-performance-management/ complete solution for microservices architecture ### Code: [examples](./client) ## Presenting ### Links - https://github.com/yaronn/blessed-contrib console dashboard visualization framework - http://stack.gl/ one of many webGL libraries for more experimental visualizations - https://threejs.org/ one of the more popular webgl libraries - https://github.com/regl-project/regl webgl library - http://fontawesome.io/ great icon library - https://fonts.google.com/ popular soution for fonts - https://github.com/darul75/ng-prettyjson show syntax highlighted json data in Angular - http://airbnb.design/building-a-visual-language/ a guide to developing a visual language by AirBnB - http://www.datavizcatalogue.com/ selection of data viz techniques - https://www.pubnub.com/developers/eon/ real time library for client visualizations - https://d3js.org/ The most popular visualization library for client development - http://christopheviau.com/d3list/gallery.html A great gallery of D3 visualizations - http://www.aviz.fr/~bbach/timecurves/ an interesting approach to representing progress - https://github.com/Netflix/Hystrix/wiki/Dashboard A robust monitoring component that is streaming data compatible - http://grafana.org/ graph library - https://github.com/artzub/blackhole.js fancy relationship visualization, good for streaming because it has a focus on changes over time - https://github.com/uber/deck.gl map visualization library - https://github.com/datacratic/data-projector experimental 3d network visualization - https://github.com/INRIA/VisualSedimentation visualization method to show accumulated changes over time - https://github.com/anvaka/ngraph.pixel#demo graph library on webgl - http://techblog.netflix.com/2016/08/vizceral-open-source.html graph library - http://epochjs.github.io/epoch/ streaming friendly charts - https://preziotte.com/partymode/ music visualizations - https://github.com/cytoscape/cytoscape.js one of the most popular and feature rich realtionship visualization libraries - http://flowingdata.com/2017/01/24/one-dataset-visualized-25-ways/ illustrating the choices of how to best visualize a dataset - https://hal-enac.archives-ouvertes.fr/hal-01021607/document paper on node-network simplification - https://googlecreativelab.github.io/anypixel/ experimental displays - https://www.kineviz.com/ a company focused on exploring difficult data visualization challenges. - https://bl.ocks.org/emeeks/9bc7e3f505b25908a26fd8045656b490 example of treemaps with history - https://github.com/airbnb/superset airBnB open source framework for visualization using Python - https://github.com/ordina-jworks/microservices-dashboard dashboard for visualing microservices ecosystems ### Code: [examples](./presenting.md) ## Components ### Links ### Code: [examples](./components.md) ## Analysis ### Links ### Code: [examples](./analysis.md) ## Workflow ### Links ## Dashboard ### Links ### Code: [examples](./dashboard.md) ## Machine Learning ### Links - https://www.tensorflow.org/ general purpose machine learning framework - http://scikit-learn.org/ Python machine learning library - https://github.com/karpathy/convnetjs deep learning in the browser or node.js - https://github.com/Yomguithereal/talisman collection of machine learning capabilities on node.js - https://github.com/NaturalNode/natural popular natural language processing library for node.js - https://cloud.google.com/ml/ machine learning in Google cloud - https://aws.amazon.com/machine-learning/ machine learning in aws - https://azure.microsoft.com/en-us/services/machine-learning/ Microsoft Azure machine learning - http://yerevann.com/a-guide-to-deep-learning/ machine learning and deep learning tutorials - http://www.skrasser.com/publications/pdf/krasser2015val.pdf detailed description of machine learning applied to malware detection - http://distill.pub/ explanations with interactive examples on machine learning concepts ## Collaboration ### Links - https://api.slack.com/incoming-webhooks send messages to slack - https://developer.atlassian.com/hipchat/getting-started send messages to hipchat ### Code: [examples](./collaboration.md) ## Exports ### Links ### Code: [examples](./exports.md) ## Use Cases ### Links - https://initialstate.com/ streaming data visualization for Internet of Things - https://lab.interactivethings.com/galaxy-of-covers/ a beautiful illustration of song and their covers as solar systems - https://mapzen.com/blog/tron-v2-visual-scale/ visual element of tron on a streaming data map - https://www.washingtonpost.com/graphics/national/police-shootings/ interactive infographic on a hot topic - http://www.darkhorseanalytics.com/portfolio-all/ professional visualizations by a specific firm - http://dataviz.com.au/ Australian data visualization group - https://www.kiln.digital/ a collection of examples of presentations intended to convince an audience - http://d3.artzub.com/wbca/ another map visualization with a lot of coordinated elements - https://mapbox.github.io/webgl-wind/demo/ map with particle movement to show directions and paths - https://www.robscanlon.com/github-wargames/ intentionally retro visualization of live data - http://www.fudgie.org/ end to end streaming data visualization for log files - https://github.com/NarrativeScience/Log.io complete and free product for watching streaming log files - https://torflow.uncharted.software/ map of TOR traffic with relationsips - https://github.com/mbtaviz/mbtaviz.github.io/ impressive multi visualization on transportation data - http://vallandingham.me/openvis_tweets/ an interesting approach to frequency visualization using tweets - http://vallandingham.me/speaker_web/ combines frequency and relationships - https://www.behance.net/gallery/11508419/Falcon-Tactical-HUD a promotional prototype by Brian Staats for Crowdstrike - http://setosa.io/#/ collection of data visualizations - http://nulldesign.jp/metrogram3d/ 3d streaming data visualization - https://vimeo.com/groups/vast2013/videos/80829138 streaming network visualization concept ### Continuing Research: - http://dataviz.tools/ a great list of non streaming specific data visualization - http://treevis.net lots of tree and treemap visualizations - https://github.com/josephmisiti/awesome-machine-learning machine learning resources ### Added after Version 1 publication: - http://ethviewer.live/ : impressive / experimental visualization of ethereum blockchain - https://github.com/cubedro/eth-netstats - http://www.r2d3.us/visual-intro-to-machine-learning-part-1/ - http://www.r2d3.us/visual-intro-to-machine-learning-part-2/