[2.0] Spread Entries group loading

[stephpy]

Spread entries at this moment are defined like that:

$entry = new Entry($component);

Component have to be fetched from database, and if we want to spread on 1000 entries, we'll have 1000 query to fetch components. That cannot stay as it is and we could be able to have only one query to fetch all component or create non-existant components.

[thomask]

So how does this work exactly?

$userIds = array(1,4, 9, 48, 112);
$coll->add(new EntryUnaware('My\Model', $userIds));

Passing an array like this doesn't work. When I iterate over the array (which in my case has length > 1000) and add each user id individually, the query count is about 3000 :S

Thanks!

[stephpy]

You should iterate. If component are not created for each users, it'll create for each one, so it's "normal" to have a lot of queries, but if components are already created, it should make only one request.

$userIds = array(1,4, 9, 48, 112);
foreach ($userIds as $userId) {
  $coll->add(new EntryUnaware('My\Model', $userId));
}

You can optimise call by fetching component from your storage: (Actual call make a CONCAT which is may not really optimized i guess).

$components = $this->getDoctrine()->getManager()
  ->getRepository('XYZ:Component')
   ->createQueryBuilder('c')
   ->where('c.model = :model')
   ->andWhere('c.identifier IN (:identifier)')
   ->setParameter('model', 'My\Model')
   ->setParameter('identifier', array(1, 4, 9) 
// not sure, may an array of serialized ids (as string). array(serialize((string) 1), .....)
   ->getQuery()
   ->getResult();

foreach ($components as $component) {
    $coll->add(new Entry($component));
}

// iterate on components, if one not exists, from initial list, use EntryUnaware.

[stephpy]

But if component exists and you have 3000 queries, it's not an expected behavior and we should fix it.

If they not exists, it may make 3000 inserts, i should look at this step to optimize it.

[thomask]

Well the components are only inserted ones, so that is ok. However, each insert in the timeline table gets its own query. Wouldn't it be possible to handle the whole insert in one query (or a couple)?

[stephpy]

Curious, it should be. Deployer call X times createAndPersist (which only persist, no flush), and then make a flush.
I did not test what doctrine does when there is X insert then a flush, i thought it makes only one INSERT.

If you deploy X actions, it should make X inserts. If you deploy 1 action, it should make 1 insert or it is a doctrine "issue", not sure to be able to fix it.

[stephpy]

see http://docs.doctrine-project.org/projects/doctrine-orm/en/2.0.x/reference/batch-processing.html#bulk-inserts, it is what is done ATM (except clear, i should implement this but it'll not fix number of INSERT).

I created an issue #61.