aigents/aigents-java

Aigents Cloud Storage

akolonin opened this issue · 0 comments

For the high-performance and high-capacity Aigents Servers supporting thousands and millions of users, we would need to change the existing storage design of the Aigents.
Currently, it involves:
A) in-memory custom graph DB https://github.com/aigents/aigents-java/blob/master/src/main/java/net/webstructor/core/Storager.java (stored in al.txt snapshots)
B) "temporal graphs" for indexing source-specific historical graph data https://github.com/aigents/aigents-java/blob/master/src/main/java/net/webstructor/data/GraphCacher.java
C) "long-term memory" storage of the object instances https://github.com/aigents/aigents-java/blob/master/src/main/java/net/webstructor/core/Archiver.java
D) cache of the web data https://github.com/aigents/aigents-java/blob/master/src/main/java/net/webstructor/self/Cacher.java
While the above work fine for single-user Aigents instances and up to few-hundred-user instances, it may not scale well if we get thousands and millions of concurrent Line Clients (such as per #25 , #26 #27 and #28 ) so the following would have to get done:

  1. Redesign and refactor the above so we use interfaces instead of classes and the implementations of those interfaces are served by factories obtained at the Body level https://github.com/aigents/aigents-java/blob/master/src/main/java/net/webstructor/agent/Body.java (see getPublisher singleton factory for instance)
  2. Choose the Graph/SQL/Object DB engine for alternative implementation of these interfaces (like Neo4J/PostgreSQL/MongoDB)
  3. Have the job done :-)