Drupal 11: Using The Finished State In Batch Processing

First we need to create a batch process that will accept the array we want to process. This is the same array as we have processed in the last two articles, but in this case we are passing the entire array to a single callback via the addOperation() method of the BatchBuilder class.$batch = new BatchBuilder();
$batch->setTitle('Running batch process.')
->setFinishCallback([BatchClass::class, 'batchFinished'])
->setInitMessage('Commencing')
->setProgressMessage('Processing...')
->setErrorMessage('An error occurred during processing.');

batch_set($batch->toArray());

Let’s say that we wanted to process a number of nodes top perform an action on them. This might be updating them in some way, or even deleting them. Instead of loading all of the nodes we want to process at the start we would start off the batch process without loading anything or passing any arguments to the addOperation() method. In this article we looked at how to control the batch flow using the finished setting in the provided batch context. This setting allows us to control the flow of the batch operations by setting a value within the batch. The finished value can be used to run open ended batch operations where we don’t need to load in all of the items we will publish at the start of the process.

Setting Up

Of course, processing numbers is simple, but let’s do something more interesting with the finished setting.Everything else about the batch process has remained the same. We are still using the same finish method that we defined in the previous articles.There is another way to set up the Batch API that will run the same number of operations without defining how many times we want to run them first. This is possible by using the “finished” setting in the batch context.Let’s create a batch process that we can run and control using the finished setting.// Keep track of progress.
$context['sandbox']['progress'] += $batchSize;

This is my preferred method of using batch operations, especially when entity types are involved. Batch operations that load in the list of entities to process work fine with a few items. The problem is that when you add 100k or even a million records to the database the batch tends to fall over before it can get started. I have converted a few batch operations to use the finished system in the last few years so I always try to start with this principle when writing batch operations.In the next article we will look at how the Batch API and the finished setting can be used to process a CSV file of any length. $batchSize = 10;

$context['finished'] = $context['sandbox']['progress'] / $context['sandbox']['max'];

Since we have only passed a single array as the argument of the batch process we only need to accept the array and the $context parameter from the batch callback.if ($context['sandbox']['progress'] <= $context['sandbox']['max']) {
$context['finished'] = 0;
}

$batch = new BatchBuilder();
$batch->setTitle('Running batch process.')
->setFinishCallback([BatchClass::class, 'batchFinished'])
->setInitMessage('Commencing')
->setProgressMessage('Processing...')
->setErrorMessage('An error occurred during processing.');

batch_set($batch->toArray());

public static function batchProcess(array &$context): void {
if (!isset($context['sandbox']['progress'])) {
$query = Drupal::entityQuery('node');
$query->accessCheck(FALSE);
$count = $query->count()->execute();

$context['sandbox']['progress'] = 0;
$context['sandbox']['max'] = $count;
}

public static function batchProcess(array $array, array &$context): void {
if (!isset($context['sandbox']['progress'])) {
$context['sandbox']['progress'] = 0;
$context['sandbox']['max'] = count($array);
}
}

public static function batchProcess(array $array, array &$context): void {
if (!isset($context['sandbox']['progress'])) {
$context['sandbox']['progress'] = 0;
$context['sandbox']['max'] = count($array);
}
if (!isset($context['results']['updated'])) {
$context['results']['updated'] = 0;
$context['results']['skipped'] = 0;
$context['results']['failed'] = 0;
$context['results']['progress'] = 0;
$context['results']['process'] = 'Finish batch completed';
}

$context['finished'] = $context['sandbox']['progress'] / $context['sandbox']['max'];
}

We can either do a simple comparison between the progress and the max value and set the finished value to be 0 if we haven’t reached the end of the processing yet.$batchSize = 100;

}

$context['finished'] = $context['sandbox']['progress'] / $context['sandbox']['max'];

All of the code here is available on the Drupal Batch Examples repository. Please let me know if you found these articles or that code useful.

• Initial batch process call. 
  • Set the max value setting to 1,000 and process 100 items of the array.
  • Update the progress count to be 100.
  • Set the value of finished to be 100/1000 (or 0.1).
  • As this number is less than 1 the process operation is called again.
• Second batch run.
  • Update the progress count to be 200,
  • Set the value of finished to be 200/1000 (0.2).
  • As this number is less than 1 the process operation is called again.
• Third batch run.
  • Update the progress count to be 300.
  • Set the value of finished to be 300/1000 (0.3).
  • As this number is less than 1 the process operation is called again.
• Skip a few iterations…
• Tenth batch run.
  • Update the progress count to be 1000.
  • Set the value of finished to be 1000/1000 (1).
  • As the number is 1 the process operation is considered complete and removed from the batch.
• The finish callback method is run, passing the results of the batch process into the callback.

public static function batchProcess(array $array, array &$context): void {

}

To fully illustrate what is going on here, let’s step through the batch process.So far in this series we have looked at creating a batch process using a form and then creating a batch class so that batches can be run through Drush. Both of these examples used the Batch API to run a set number of items through a set number of process function callbacks. When setting up the batch run we created a list of items that we wanted to process and then split this list up into chunks, each chunk being sent to a batch process callback.When we first start off the batch operation the $context array contains the following items. Remember that this array is passed by reference, so any changes made to it will be visible to the code that called it.The key here is setting a value of less than 1. If we do this then the batch will not remove the operation from the queue and the batch process will simply call the process method again. If we select this method of operation then we need to ensure that we keep track of the progress of the batch operation so that we can say if we have finished the batch operation.All the code you see here is available in the Drupal Batch Examples repository of GitHub, with the finished example appearing as one of the available sub modules. Feel free to use this as the basis of your own batch process methods.

Conclusion

Array(
[sandbox] => Array()
[results] => Array()
[finished] => 1
[message] =>
)

Putting this all together gives us a batchProcess method that looks like this.This is the third article in a series of articles about the Batch API in Drupal. The Batch API is a system in Drupal that allows data to be processed in small chunks in order to prevent timeout errors or memory problems.When the batch process is first called the “sandbox” part of the $context array is empty, and we can use this array to keep track of the progress and how many items we have left to process.