VelocityDB & VelocityGraph

[AutoIncrement] field attribute added. Use it to automatically get a unique number (sequence from 1 normally) generated for every instance of a class to be stored in a field within a class, such as in:

  public class Person : OptimizedPersistable
  {
    string firstName;
    string lastName;
    ushort age;
    UInt64 idNumber;
    Person bestFriend;
    [AutoIncrement]
    UInt64 autoIncrement;
    SortedSetAny<Person> friends;

For ServerClientSession, only invalidate pages updated by another transaction instead of invalidating an entire Database. Updated session constructor parameter.

Fix directory compare and transaction id increment in VelocityDBServer.

Make strong object references cache default in SessionNoServer constructor and switch to weak object reference cache when memory available is low, switch back to strong references when more memory is available.

Rerun benchmark on a slightly faster PC, looks great! This time with compression turned on.

Enable strong object reference cache with ServerClientSession constructor.

By default, strong references are now used instead of weak references.

A client test case ran more 1000 times faster with this option. Caching is improved.

1) Where does it cache data? In memory on server side or client side? If I am using such code

var session = getCachedSession();
session.BeginRead();
var data = GetData();
session.Commit();
return data;

then next GetData() always opens objects from server disk (even with CacheEnum.Yes).

If the code just like that:

var session = getCachedSession(); //long read transaction
return GetData();

then GetData() immediately gets data from client memory. I don't want to store data in client's memory. How can I cache data only in the memory of the server till first update of related pages? What about caching of system databases (schema, locations etc)?

2) What is proper usage of ServerClientSession in multiple threads (in one process)? I see, the class itself is not thread-safe, and creation of an instance is expensive. Is there any way to make its singleton instance thread-safe (without lock(session) )? In the moment I am manually maintaining a pool of several initialized ServerClientSession using ConcurrentStack; each thread pops (creates if needed) and pushes back own instance with separate transaction block (BeginRead/Update - Commit block). Maybe is there better approach that we could use?

Replace a Queue with a RingBuffer (as in Ndisruptor) in server request management.

Benchmark Files for VelocityDB

VelocityDBServerBenchmark.cs (1.2KB)

VelocityDBBenchmark.cs (3.2KB)

Use BufferedStream for network writes to reduce networks packets

See http://www.tsjensen.com/blog/post/2014/01/21/BufferedStream-Improves-NET-Sockets-Performance.aspx

Can you provide autoincrementing fields of numeric types in persistent objects by implementing a field attribute like [AutoIncrement] ? One will be able to add that attribute to field with

 [Index][UniqueConstraint]

but the field value is automatically incremented when object is persisted.

It must be retroactive if there are objects with such fields persisted already. For example if we had imported the objects with specified fields values with compiled class without that attribute, then next time after we put that attribute and compile it, Session must get the last value in indexed fields and increment it for new object.

Fix a recursive issue in recovery

Add databasePerType parameter (default == true) to SessionBase.AllObjects for faster lookups

foreach (var obj in session.AllObjects<DomainObject>().Take(10))

Console.WriteLine("id: {0}, products: {1}, likes: {2}, comments: {3}", obj.Id, obj.Products.Count, obj.Likes.Count, obj.Comments.Count);

If you have a lot of different types of objects and many of each, this option will make it fast to find a certain type of object. Otherwise we have to examine each page in every database in order to find all objects of a certain type. If you use this option (on by default) you must persist your objects using the recommended Persist API, i.e.

var look = new DomainObject(session, i);

We have the issues with slow usage of ServerClientSession in local network. Version of VelocityDb is 3.13.1. Toggled TcpClient.NoDelay does not help.

First of all, let me to show some benchmarks made by Database Benchmark 2.0.3 . I've just added benchmark for ClientServerSession leading to localhost (same PC with client) and second one leading to remote host (in same network as client).

Write speed

Read speed

Secondary read speed

As you can see, result of ServerClientSession connecting to remote host is very slow. Do you have any benchmark results for ServerClientSession?

There is the source code of an example, the usage of session and classes in it are similar to my models in web application. The program also runs very slow on the network (especially for read of 10 objects on wire). There are the profiler results. You can see that the huge bottleneck occurs in network code (like Socket.Receive). Elapsed time in hundreds milliseconds is not so acceptable.

What can you suggest? Or maybe is there some bottleneck in requests loop in VelocityDbServer or something like that?

We changed TcpClient.NoDelay = true to speed up client/server communication.

Kevin Bacon Numbers, calculating degree of seperation to all other actors and actresses!

Social network graph data, finding connections forming triangles!

World Cities sample: Now with an associative click interface

Sample:

AllObjects<NotSharingPage> all = db.AllObjects<NotSharingPage>();

List<NotSharingPage> notSharingPageList = all.Skip(100).ToList();

Download it.

We are starting with baby steps...posts any questions or answers you have here! Press Login button and create a user name for yourself then you are ready to comment/post, upload code/videos, whatever you need to do.

Go back to VelocityDB or VelocityGraph when you are done/bored.