A collection of resources on dynamic/streaming/temporal/evolving graph processing systems, databases, data structures, datasets, and related academic and industrial work.
To add, remove or change things on the list: please submit a pull request to the GitHub repository.
- Aspen [C++]
- Streaming graph processing system based on compressed purely-functional trees
- Paper
- Differential Dataflow [Rust]
- General data-parallel programming framework suitable for processing large dynamic graphs
- Paper
- GraphBolt [C++]
- Streaming graph processing system that provides BSP guarantees
- Paper
- GraphJet [Java]
- Streaming graph processing system for generating content recommendations at Twitter
- Paper
- GraphOne [C++]
- Storage engine for streaming graph data
- Paper
- GreyCat [Java]
- Many-world temporal graph data store
- Paper
- GoFFish[Java]
- Multi-source streaming graph processing system
- KickStarter[C++]
- Streaming graph processing system optimising for path-based/monotonic graph algorithms
- Paper
- LLAMA [C++]
- CSR-based storage engine and streaming graph processing system
- Paper
- PCSR [C++]
- CSR-inspired, packed memory array based data structure for streaming graphs
- Paper
- PPCSR [C++]
- Parallel and NUMA-aware implementation of PCSR data structure for streaming graphs
- Raphtory [Scala]
- Snapshot-based distributed streaming graph processing system
- Paper
- STINGER [C/C++]
- Data structure for streaming graphs based on linked lists of blocks
- Paper
- GraphTides [Java/TypeScript]
- Framework for evaluating streaming graph processing platforms
- Paper
Check Julian Shun's list for a further reference and an extensive collection of related papers.
STINGER: Spatio-Temporal Interaction Networks and Graphs (STING) Extensible Representation (2009)
STINGER: High Performance Data Structure for Streaming Graphs (HPEC 2012)
Packed Compressed Sparse Row: A Dynamic Graph Representation (HPEC 2018)
GraphOne: A Data Store for Real-time Analytics on Evolving Graphs (FAST 2019)
GraphTinker: A High Performance Data Structure for Dynamic Graph Processing (IPDPS 2019)
CSR++: A Fast, Scalable, Update-Friendly Graph DataStructure (OPODIS 2020)
LiveGraph: A Transactional Graph Storage System With Purely Sequential Adjacency List Scans (VLDB 2020)
A Parallel Packed Memory Array to Store Dynamic Graphs (ALENEX 2021)
STINGER: Spatio-Temporal Interaction Networks and Graphs (STING) Extensible Representation (2009)
On Querying Historical Evolving Graph Sequences (VLDB 2011)
STINGER: High Performance Data Structure for Streaming Graphs (HPEC 2012)
Facilitating Real-Time Graph Mining (CloudDB 2012)
Kineograph: Taking the Pulse of a Fast-Changing and Connected World (EuroSys 2012)
Differential Dataflow (CIDR 2013)
Efficient Snapshot Retrieval over Historical Graph Data (ICDE 2013)
Naiad: A Timely Dataflow System (SOSP 2013)
Chronos: A Graph Engine for Temporal Graph Analysis (EuroSys 2014)
Towards Large-Scale Graph Stream Processing Platform (WWW 2014)
CellIQ : Real-Time Cellular Network Analytics at Scale (NSDI 2015)
DISTINGER: A Distributed Graph Data Structure for Massive Dynamic Graph Processing (Big Data 2015)
ImmortalGraph: A System for Storage and Analysis of Temporal Graphs (Transactions on Storage 2015)
LLAMA: Efficient Graph Analytics Using Large Multiversioned Arrays (ICDE 2015)
Real-time Analytics for Fast Evolving Social Graphs (CCGrid 2015)
GraphIn: An Online High Performance Incremental Graph Processing Framework (Euro-Par 2016)
GraphJet: Real-Time Content Recommendations at Twitter (VLDB 2016)
Storing and Analyzing Historical Graph Data at Scale (EDBT 2016)
Synergistic Analysis of Evolving Graphs (TACO 2016)
Time-Evolving Graph Processing at Scale (GRADES 2016)
Tornado: A System For Real-Time Iterative Analysis Over Evolving Data (SIGMOD 2016)
Towards a Distributed Large-Scale Dynamic Graph Data Store (IPDPSW 2016)
Automatic Algorithm Transformation for Efficient Multi-Snapshot Analytics on Temporal Graphs (VLDB 2017)
KickStarter: Fast and Accurate Computations on Streaming Graphs via Trimmed Approximations (ASPLOS 2017)
Auxo: A Temporal Graph Management System (BDMA 2018)
Distributed Evaluation of Subgraph Queries Using Worst-case Optimal Low-Memory Dataflows (VLDB 2018)
GraPU: Accelerate Streaming Graph Analysis through Preprocessing Buffered Updates (SOCC 2018)
GraphBolt: Dependency-Driven Synchronous Processing of Streaming Graphs (EuroSys 2019)
Low-Latency Graph Streaming Using Compressed Purely-Functional Trees (PLDI 2019)
GraphOne: A Data Store for Real-time Analytics on Evolving Graphs (FAST 2019)
LiveGraph: A Transactional Graph Storage System With Purely Sequential Adjacency List Scans (VLDB 2020)
Raphtory: Streaming analysis of distributed temporal graphs (FGCS 2020)
RisGraph: A Real-Time Streaming System for Evolving Graphs (2020)
Tegra: Efficient Ad-Hoc Analytics on Time-Evolving Graphs (NSDI 2021)
The Ubiquity of Large Graphs and Surprising Challenges of Graph Processing (VLDB 2018)
Practice of Streaming and Dynamic Graphs: Concepts, Models, Systems, and Parallelism (2020)
Note: Very small datasets are not included in the list.
| Category | Data Source | Name | Type | Vertex Meaning | Edge Meaning | Size (|V|) | Volume (|E|) | Edge Weight | Edge Timestamps | Multiple Edges | Source/Link |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Authorship network | Wikipedia | edit-enwiki | Bipartite, undirected | User, article | Edit | 50,757,442 | 572,591,272 | N | Y | Y | http://konect.cc/networks/edit-enwiki/ |
| Citation network | DBLP | dblp-cite | Unipartite, directed | Publication | Citation | 12,590 | 49,759 | N | Y | N | http://konect.cc/networks/dblp-cite/ |
| Co-citation network | arXiv | ca-cit-HepPh | Unipartite, undirected | Author | Co-citation | 28,093 | 4,596,803 | N | Y | Y | http://konect.cc/networks/ca-cit-HepPh/ |
| Co-citation network | arXiv | ca-cit-HepTh | Unipartite, undirected | Author | Co-citation | 22,908 | 2,673,133 | N | Y | Y | http://konect.cc/networks/ca-cit-HepTh/ |
| Communication network | facebook-wosn-wall | Unipartite, directed | User | Wall post | 46,952 | 876,993 | N | Y | Y | http://konect.cc/networks/facebook-wosn-wall/ | |
| Communication network | Linux kernel mailing list | lkml-reply | Unipartite, directed | Person | Reply | 63,399 | 1,096,440 | N | Y | Y | http://konect.cc/networks/lkml-reply/ |
| Computer network | Internet topology | topology | Unipartite, undirected | Autonomous system | Connection | 34,761 | 171,403 | N | Y | Y | http://konect.cc/networks/topology/ |
| Feature network | BibSonomy | bibsonomy-2ti | Bipartite, undirected | Tag, publication | Assignment | 972,120 | 2,555,080 | N | Y | Y | http://konect.cc/networks/bibsonomy-2ti/ |
| Feature network | CiteULike | citeulike-ti | Bipartite, undirected | Tag, publication | Assignment | 885,046 | 2,411,819 | N | Y | Y | http://konect.cc/networks/citeulike-ti/ |
| Feature network | Delicious | delicious-ti | Bipartite, undirected | Tag, URL | Tag assignment | 38,289,740 | 301,183,605 | N | Y | Y | http://konect.cc/networks/delicious-ti/ |
| Feature network | Google+ | stevgong_attribute | Bipartite, undirected | User, attribute | Attribute-link | 10,442,421 | 20,592,962 | N | Y | N | http://konect.cc/networks/stevgong_attribute/ |
| Feature network | munmun_twitterex_ti | Bipartite, undirected | Hashtag, URL | Co-occurrence | 1,502,611 | 2,635,885 | N | Y | Y | http://konect.cc/networks/munmun_twitterex_ti/ | |
| Human contact network | MIT | mit | Unipartite, undirected | Person | Contact | 96 | 1,086,404 | N | Y | Y | http://konect.cc/networks/mit/ |
| Hyperlink network | Wikipedia | link-dynamic-dewiki | Unipartite, directed | Article | Reference | 2,166,669 | 86,337,879 | N | Y | N | http://konect.cc/networks/link-dynamic-dewiki/ |
| Hyperlink network | Wikipedia | link-dynamic-frwiki | Unipartite, directed | Article | Reference | 2,212,682 | 59,008,831 | N | Y | N | http://konect.cc/networks/link-dynamic-frwiki/ |
| Hyperlink network | Wikipedia | link-dynamic-itwiki | Unipartite, directed | Article | Reference | 1,204,009 | 34,826,283 | N | Y | N | http://konect.cc/networks/link-dynamic-itwiki/ |
| Hyperlink network | Wikipedia | link-dynamic-nlwiki | Unipartite, directed | Article | Reference | 1,039,252 | 20,070,561 | N | Y | N | http://konect.cc/networks/link-dynamic-nlwiki/ |
| Hyperlink network | Wikipedia | link-dynamic-plwiki | Unipartite, directed | Article | Reference | 1,033,050 | 25,026,208 | N | Y | N | http://konect.cc/networks/link-dynamic-plwiki/ |
| Hyperlink network | Wikipedia | link-dynamic-simplewiki | Unipartite, directed | Article | Reference | 100,312 | 1,627,472 | N | Y | N | http://konect.cc/networks/link-dynamic-simplewiki/ |
| Interaction network | LastFm | lastfm_band | Bipartite, undirected | User, band | Listening | 175,069 | 19,150,868 | N | Y | Y | http://konect.cc/networks/lastfm_band/ |
| Interaction network | LastFm | lastfm_song | Bipartite, undirected | User, song | Listening | 1,085,612 | 19,150,868 | N | Y | Y | http://konect.cc/networks/lastfm_song/ |
| Online contact network | Stack Exchange | sx-askubuntu | Unipartite, directed | User | Answer/comment | 159,316 | 964,437 | N | Y | Y | http://konect.cc/networks/sx-askubuntu/ |
| Online contact network | Stack Exchange | sx-mathoverflow | Unipartite, directed | User | Answer/comment | 24,818 | 506,550 | N | Y | Y | http://konect.cc/networks/sx-mathoverflow/ |
| Online contact network | Stack Exchange | sx-stackoverflow | Unipartite, directed | User | Answer/comment | 2,601,977 | 63,497,050 | N | Y | Y | http://konect.cc/networks/sx-stackoverflow/ |
| Online contact network | Stack Exchange | sx-superuser | Unipartite, directed | User | Answer/comment | 194,085 | 1,443,339 | N | Y | Y | http://konect.cc/networks/sx-superuser/ |
| Online contact network | higgs-twitter | Unipartite, directed | User | Retweet/mention/reply | ? | 563,069 | N | Y | Y (?) | https://snap.stanford.edu/data/higgs-twitter.html | |
| Online contact network | munmun_twitterex_at | Unipartite, directed | User | Mention | 2,919,613 | 12,887,063 | N | Y | Y | http://konect.cc/networks/munmun_twitterex_at/ | |
| Online contact network | Wikipedia | elec | Unipartite, directed | User | Vote | 7,118 | 103,675 | N | Y | N | http://konect.cc/networks/elec/ |
| Online contact network | Wikipedia | wikiconflict | Unipartite, undirected | User | Edit conflict | 118,100 | 2,917,785 | Y | Y | Y | http://konect.cc/networks/wikiconflict/ |
| Online social network | Bitcoin | soc-bitcoin | Unipartite, directed | Member | Trust/distrust | 24,575,382 | 122,948,162 | ? | Y | N | https://networkrepository.com/soc-bitcoin.php |
| Online social network | Digg | digg-friends | Unipartite, directed | User | Vote | 279,630 | 1,731,653 | N | Y | N | http://konect.cc/networks/digg-friends/ |
| Online social network | facebook-wosn-links | Unipartite, undirected | User | Friendship | 63,731 | 817,035 | N | Y | N | http://konect.cc/networks/facebook-wosn-links/ | |
| Online social network | Flickr | flickr-growth | Unipartite, directed | User | Friendship | 2,302,925 | 33,140,017 | N | Y | N | http://konect.cc/networks/flickr-growth/ |
| Online social network | Google+ | stevgong_social | Unipartite, directed | User | Interaction | 28,943,739 | 462,994,069 | N | Y | N | http://konect.cc/networks/stevgong_social/ |
| Online social network | YouTube | youtube-u-growth | Unipartite, undirected | User | Friendship | 3,223,589 | 9,375,374 | N | Y | N | http://konect.cc/networks/youtube-u-growth/ |
| Rating network | Amazon | rec-amz-Books | Bipartite, undirected | User, product | Rating | 10,356,390 | 22,507,155 | Y | Y | N | https://networkrepository.com/rec-amz-Books.php |
| Rating network | Epinions | epinions-rating | Bipartite, undirected | User, product | Rating | 876,252 | 13,668,320 | N | Y | Y | http://konect.cc/networks/epinions-rating/ |
| Rating network | Netflix | netflix | Bipartite, undirected | User, movie | Rating | 497,959 | 100,480,507 | Y | Y | N | http://konect.cc/networks/netflix/ |
| Rating network | Yahoo | yahoo-song | Bipartite, undirected | Person, song | Rating | 1,625,951 | 256,804,235 | Y | Y | N | http://konect.cc/networks/yahoo-song/ |
This list was compiled by Domagoj Margan with help and resources from the systems community.