Zim Server Performance Tuning
After running your Zim applications for a while, it will probably be needed to re-configure Zim Server to provide better performance. Basically, three things can be taken into account:
1) Maximize the use of shared memory by allocating as many buffers and areas as possible;
2) At the same time, balance the use of shared memory so that it does not exceed system defined limits and creates overheads thus reducing Zim Server performance;
3) Reduce the amount of input and output operations.
These three factors are intrinsically related and are expressed through configuration options in zimconfig.srv, the Zim Server configuration file.
Below, some hints on the effects of relevant options for Zim Server performance. Additional information is also found in the specific options.
Checkpoint Buffers and Checkpoint Transactions
These options define the size of the checkpoint file, that is, either the number of blocks or transactions that Zim Server will process before physically committing to disk. Both refer to transactions that were effectively committed, that is, transactions successfully finished.
Because of Zim Server’s optimized checkpoint mechanism, only the last occurrence of a block is written to disk. This leads to the idea that the larger the checkpoint file, the more efficient Zim Server will be because less input and output will be accomplished. On the other hand, processing more blocks takes more time and Zim Server will spend less time to take care of on-going transactions.
Another issue that should be taken into account is that the bigger these options, the more time Zim Server will wait until a checkpoint file is produced and then committed to disk. If the Zim application is small or the number of users is also small, using large numbers might force Zim Server to wait several minutes before physically updating the database.
Therefore, there is a balance on these two options. To make things easier to configure, try setting Checkpoint Transactions to a very large number, like 100000, and only play with Checkpoint Buffers.
For small databases, try the following:
checkpoint buffers 1000 % small checkpoint file
checkpoint transactions 100000 % never gets to this
The above options tells Zim Server to physically update the database files after 1000 blocks are committed or after 100000 transactions are committed. The latter will probably never happen (because of the small database activity), but 1000 blocks are easily achieved.
Therefore, even though checkpoints will be very frequent, Zim Server will still be efficient because of the low activity at the database.
For larger databases, try these options:
checkpoint buffers 10000 % medium size checkpoint file
checkpoint transactions 100000 % never gets to this
Similarly, this will take more time to checkpoint but checkpoints will be more frequent because of bigger database activity but still letting Zim Server be efficient.
In the same line, bigger values for Checkpoint Buffers can be chosen, like 50000, 100000 or even bigger. Testing different values can lead to a different Zim Server performance.
Maximum Data Blocks
This option tells Zim Server how many blocks of data will be kept in shared memory at the same time. It dramatically affects Zim Server efficiency because it greatly optimizes the number of input and output operations.
Therefore, the larger this option, the more efficient Zim Server will be.
Care should be taken because it uses a big amount of shared memory: if not enough real memory is available, then the operating system will start swapping in and out in order to accommodate the need for shared memory and performance will be significantly reduced.
Maximum Databases
This tells Zim Server how many Zim databases will be processed at same time. The default is 3, but keeping it to the minimum necessary will reduce the amount of shared memory needed.
Maximum Tables
Indicates how many Zim tables (entity sets and relationships with fields) will be processed for all databases. Try keeping this number to the minimum possible so that less shared memory is used.
Maximum Users
This is the maximum number of users allowed to connect to Zim Server (up to the number of Zim licences purchased). Keep this number to the minimum needed to save share memory.
Object Sleep, Object Timeout, Pending Sleep and Pending Timeout
These parameters refer to the time to wait and time-out times when trying to acquire various different kinds of Zim Server internal objects. The lower the values, the faster Zim Server will react in order to resolve the problem (usually by deadlocking) thus allowing other users to proceed. Try using the values suggested in the configuration files presented below.
Scatter Table Entries
The number of entries refer to the maximum sizes of transactions in blocks that all users connected to Zim Server can perform at the same time. If there are 10 users connected but only 5 performing transactions, only these ones will count. For these, if the average number of blocks processed is 500, then the number of entries used will be 2,500 (as soon as a transaction is finished, either by successfully committing it or by aborting it, all entries used in that transaction are released).
This number cannot be smaller than Scatter Table Links. A good approach is to calculate 3 to 5 times that number.
Scatter Table Links
This option sets the maximum size for a transaction in blocks, that is, it allows transactions to process different blocks of data up to this limit. For instance, an ADD statement creating a very large Zim file (for example, 500 MBytes) will require this parameter to be at least 500,000. However, such commands are not very common on a daily basis which leads to the idea of having a specific configuration for large operations.
The larger this number, the more efficient Zim Server will be.
A good reference on how to establish this number is to multiply Maximum Data Blocks by two or three. In fact, this parameter cannot be smaller than Maximum Data Blocks.
Small Size Database Configuration File
This is a suggestion of a Zim Server configuration file for a small database. Replace some parameters indicated within “<>” by real numbers.
blocks per read 8
buffers per user 1000
checkpoint buffers 1000
checkpoint transactions 100000
datablock timeout 1
files per transaction 20
first file to reclaim 100
maximum blocks per user 2000
maximum data blocks 20000
maximum databases
maximum file locks 200
maximum locks 10000
maximum record groups 2000
maximum tables maximum users
object sleep 1
object timeout 1
pending sleep 5
pending timeout 5
processors 1
quick locks 10
reclaim object size 1000
record group size 256
scatter table entries 70000
scatter table links 50000
scatter table size 100
secondary lock group size 20
secondary lock groups 7500
server port number 6002
Medium Size Database Configuration File
This is a suggestion of a Zim Server configuration file for a medium size database. Replace some parameters indicated within “<>” by real numbers.
blocks per read 8
buffers per user 1000
checkpoint buffers 10000
checkpoint transactions 100000
datablock timeout 1
files per transaction 20
first file to reclaim 100
maximum blocks per user 2000
maximum data blocks 50000
maximum databases
maximum file locks 200
maximum locks 10000
maximum record groups 2000
maximum tables maximum users
object sleep 1
object timeout 1
pending sleep 5
pending timeout 5
processors 1
quick locks 10
reclaim object size 1000
record group size 256
scatter table entries 150000
scatter table links 100000
scatter table size 100
secondary lock group size 20
secondary lock groups 7500
server port number 6002
Large Size Database Configuration File
This is a suggestion of a Zim Server configuration file for a large size database. Replace some parameters indicated within “<>” by real numbers.
blocks per read 8
buffers per user 1000
checkpoint buffers 50000
checkpoint transactions 100000
datablock timeout 1
files per transaction 20
first file to reclaim 100
maximum blocks per user 2000
maximum data blocks 100000
maximum databases
maximum file locks 200
maximum locks 20000
maximum record groups 4000
maximum tables maximum users
object sleep 1
object timeout 1
pending sleep 5
pending timeout 5
processors 1
quick locks 10
reclaim object size 1000
record group size 256
scatter table entries 800000
scatter table links 250000
scatter table size 100
secondary lock group size 20
secondary lock groups 7500
server port number 6002