45 ways to speed up firebird database

45 ways to optimize
firebirdSQL
Fabio Codebue
Firebird Foundation
Committe Member

WHERE OPTIMIZE
•Hardware optimization
•Firebird server configuration
optimization
•Programming optimization
•SQL optimization

HARDWARE OPTIMIZATION
•Put database to SSD
• Put your database on SSD.
• SSD drive provides much better random IO than
traditional drives.
• Random IO is critical for reading and
writing data distributed through
big database file - the majority of
database operations require
intensive parallel random IO.

•Use RAID 10
• If you use RAID1 or RAID5, consider RAID10
• RAID10 it is 15-25% faster.

•Use BBU
• If you are using RAID controller, check that it has Backup
Battery Unit (BBU) installed and operational – some vendors
do not provide BBU by default.
• Without BBU, the controller disables the cache, and RAID
works very slow, even slower than usual SATA drives.
• Usually, you can check BBU status in the RAID configuration
tool.

•Set write cache to write-back
• If you are using RAID controller with installed BBU (and
server with UPS), check that its cache is set to write-back
(not write-through).
• «Write-back» enables write cache of the controller.

•Enable read cache
• If you use RAID controller
check that it has enabled read cache

•Check disk subsystem
• Check your drives for bad blocks and other hardware
problems (including overheating).
• Hardware problems can significantly decrease IO
performance and lead to database corruptions.

FIREBIRD CONF OPTIMIZATION
Firebird 2.5
•Use SuperClassic or Classic in Firebird
• If you use Firebird 2.5 SuperServer with many
connections, try to use SuperClassic or Classic, they can
scale better by using all cores of CPU.

FIREBIRD OPTIMIZATION
Firebird 3.x
•Use SuperServer 3.0
• If you use Classic or SuperClassic in 2.5, consider
migration to Firebird 3.0 SuperServer
• Now it can use multiple cores and combine it with the
advantages of the shared cache.

•Increase page buffers cache
• Increase the size of page buffers cache (parameter
DefaultDBCachePages) from the default values.
• For 2.5 SuperServer we recommend 10000 pages,
• For 3.0 SuperServer – 50000 pages
• For Classic and SuperClassic – from 256 to 2048 pages.
• However, don't set page buffers cache value too high – cache
synchronization has its cost, and the idea to put all database
into RAM by tuning this value will not work.

•Increase memory size for sort operations
• Increase the value of TempCacheLimit parameter in
firebird.conf
• It specifies the size of the cache of the temporary space for
sorting.
• Default values are too low
• 8Mb for Classic and 64Mb for SuperServer
• Use at least
• 64Mb for Classic
• 1Gb for SuperServer and SuperClassic.

•Set Forced Writes Off – ATTENTION!!!
• If you have intensive insert or update activity
• You can check it with HQbird MonLogger,
• If you have UPS and replication installed to protect from
hardware failures, consider to set Forced Writes settings to
OFF, it can increase speed of write operations up to 3 times.

•Increase number of hash slots for
Classic/SuperClassic
• Increase the value of LockHashSlots parameter for Classic
and SuperClassic from the default 1009 to some big prime
number (30011, for example)
• It will decrease queues in the internal locking mechanism

Firebird 2.5
•Use CPU Affinity for Super Server
•Set CPUAffinity parameter to the value equal to the
number of databases in use:
•SuperServer in 2.5 can use different CPU cores to
process requests for the certain databases.
• CpuAffinityMask = n
The value is taken from a bit map in which each bit represents a CPU.
Thus, to use only the first processor, the value is 1.
To use both
CPU 1 and CPU 2, the value is 3.
To use CPU 2 and CPU 3, the value is 6.

•Use fast drive for temp space
• Set the first part of TempDirectory parameter in
firebird.conf to the fast disk – SSD or RAM drive.
• It will decrease the time of big sortings – for example
when the database is being restored.
TempDirectories = c:temp;d:temp

•Store backups on another drive
• Store database backups on the dedicated physical drive
(RAID).
• It will separate read and write IO during backup, and
increase backup speed and decrease load for the main
drive.
• It is especially important when backups are taken while
users are working with the database.

PROGRAMMING OPTIMIZATION
•Deactivate indices for bulk inserts
• If you insert or update many records (more than 25% of
the table), deactivate indices for the table where records
are inserted and reactivate them after insert or update.
• The index rebuild operation can be faster than many
updates of the index.

•Use Global Temporary Tables for fast
inserts
• To speed up inserts and updates, use Global Temporary
Tables for bulk inserts of the large recordsets,
• and then transfer records into the permanent table.
GTT TABLE
Massive
insert
process

•Avoid unnecessary indices
• Use fewer indices for tables with intensive inserts and
updates.
• Each index adds significant overhead for insert, update,
delete, and garbage collection operations
• There could be 3-4 additional page reads and writes when
the single record is being inserted, updated, deleted,
cleaned for each index.

•Replace UDFs with embedded functions
calls
• Replace UDF calls with embedded functions calls.
• Many embedded functions were added in the recent
versions of Firebird, which offer functionality previously
available only in UDF libraries.
• Replace such functions where possible, since embedded
functions work up to 3 times faster than UDFs.

•Use read-only transactions for read
operations
• Use read-only transactions for operations which do not
change record (i.e., SELECTs) with isolation mode = read
committed.
• Such transactions do not retain record versions from the
garbage collection, and can run indefinitely
• They do not affect database performance.

•Use short write transactions and get rid
of ALL long-running
• Use short writeable transactions (for operations INSERT,UPDATE,
DELETE).
• The shorter writeable transaction is, the better.
• The short transactions retain proportionally less number of record
versions from garbage collection than long-running. Unfortunately,
even the single long-running transaction (from the development tool
left open, for example) can screw the good effect of all other short
writeable transaction.
• That's why you need to monitor long-running transaction and fix the
appropriate places in the source code.

•Avoid long record chains
• Avoid situations when one record has many record
versions
• Firebird works much slower with long record chains. (to
see how many record versions some tables has, and what
is the longest record chain you can use HQbird IBAnalyst
tool, tab Tables, sort on "Max Version").
• Use the combination of inserts and scheduled delete of
old records instead of multiple updates of the same
record.

SQL OPTIMIZATION
•Use PREPARE correctly
• Use prepared statements to run SQL queries where only
parameters are changed
for example, make prepare before the loop of such queries.
• Prepare can take significant time (especially for big
tables), and preparing the query only once will greatly
increase the overall performance.

SQL OPTIMIZATION
•Don't COMMIT too often during bulk
insert/update operation
• In the case of bulk INSERT/UPDATE/DELETE operation,
don't commit the transaction after each change
• It can happen if you are using auto commit option in your
database driver
• Commit transactions at least after 1000 operations or
more.
• Each transaction commit runs several read/write IO
operations against the database, that's why often
commits decrease database performance.

SQL OPTIMIZATION
•"Turn off" indices if you are using IN
with many constants
• If you are using construction
WHERE fieldX IN (Constant1, Constant2,… ConstantN)
and there is an index on fieldX, Firebird will use an index as
many times as many constants are in the IN list.
Disable index search by turning fieldX into expression +0:
WHERE fieldX+0 IN (Constant1, Constant2,… ConstantN), or,
for strings, use fieldX||''

SQL OPTIMIZATION
•Replace IN with JOIN
• Avoid using queries with nested WHERE IN(SELECT...
WHERE IN (SELECT.. WHERE IN() ))
• It can confuse Firebird optimizer.
• Transform nested INs into joins.

SQL OPTIMIZATION
•Use LEFT JOIN in the correct way
• If you are using LEFT OUTER joins, explicitly put tables in
the join from the smallest one to the largest one.

SQL OPTIMIZATION
•Limit fetch of SELECT queries
• Always try to limit the large output for SELECT queries
with FIRST… SKIP or ROWS clauses.
• If the query is not designed specifically as a report (which
requires all records to be printed/exported), usually it is
enough to show top 10-100 records.
• Fetch only necessary records.

SQL OPTIMIZATION
•Specify less number of columns in
SELECT with ORDER BY/GROUP BY
• Reduce the number of columns and their summary width in
queries with ORDER BY/GROUP BY both in SELECT part (i.e.,
fields to be shown) and in the ORDER BY clause.
• Firebird merges columns from SELECT and ORDER BY/GROUP
BY clauses and sorts them in memory (or, if memory is not
enough, on the disk).
• A long VARCHAR in SELECT, the size of the sort files can be
really large (many gigabytes).

SQL OPTIMIZATION
•Use derived tables to optimize SELECT
with ORDER BY/GROUP BY
• Another way to optimize SQL query with sorting is to use
derived tables to avoid unnecessary sort operations.
SELECT T.FIELD_KEY, T.FIELD1, T.FIELD2, ... T.FIELD_N
FROM (SELECT FIELD_KEY FROM T ORDER BY FIELD2)
T2
JOIN T ON T.FIELD_KEY = T2.FIELD_KEY
SELECT FIELD_KEY, FIELD1, FIELD2, ... FIELD_N
FROM T
ORDER BY FIELD2

SQL OPTIMIZATION
•Store short strings in VARCHAR, large in
BLOBs
• To store short character data, use VARCHARs,
• To store long texts, use BLOBs.
Varchars are faster for the small pieces of data because they
are stored in the record, and the whole record is read during
the same IO cycle, and if record size is less than 2/3 of the
database page size, the whole record is stored on the same
database page.
BLOBs are stored outside of the record, and require the
additional round of IO to read it, and they show the advantage
with reading and writing long strings.

SQL OPTIMIZATION
•Exclude BLOB columns from the large
SELECTs
• Exclude BLOB columns from the large SELECTs.
• Use a kind of late binding with sub-selects to selectively
show information from BLOBs
(for example, show the content of the document).

SQL OPTIMIZATION
•Use BIGINT for primary and unique keys
• Use BIGINT type for auto-incremented primary and
unique keys and for identifiers of all types.
• Operations with BIGINT are the fastest
• BIGINT has enough capacity to store almost all data
ranges.

SQL OPTIMIZATION
•Don't use VARCHARs for keys
• Don't use VARCHAR for identifiers unless it is really
necessary
• Operations with them are far less effective than with
integer columns.
• Especially avoid GUIDs, as identifier
due to the random distribution of GUID values
INSERT/UPDATE operations with Primary/Unique Keys
GUIDs can be 20 times slower than with integers.

SQL OPTIMIZATION
•Recalculate indices statistics
• Recalculate indices statistics regularly.
• Update indices statistics for the tables with frequent or
massive changes with command SET STATISTICS
• It allows Firebird optimizer to choose better SQL plans.
• HQbird Firebird DataGuard can perform such recalculation of
indices statistics automatically according to the desired
schedule (usually once a week).

SQL OPTIMIZATION
•Use connection pool
• If database connections to Firebird database are short use
connection pool
typical for web-app in PHP use
function ibase_pconnect
instead of
ibase_connect

SQL OPTIMIZATION
Firebird 3.0
•Use LINGER option
• Database connections are short
• LINGER option to keep cache active during the specified amount
of time
• It will keep frequently used pages in the cache even if there will
be no other connections.
• For example, ALTER DATABASE SET LINGER TO 60 will keep the
cache for 60 seconds after the end of the last connection.

SQL OPTIMIZATION
Firebird 3.0
•Use HASH JOINs
• Case the of joining big and small tables, HASH JOIN could be
much faster than normal join which uses «nested loop» with
index.
• To make Firebird optimizer to use HASH join, use +0 in the
join condition:
T1 JOIN T2 ON T1.FIELD1+0 = T2.FIELD2+0

SQL OPTIMIZATION
Firebird 3.0
•Mark appropriate PSQL functions as
DETERMINISTIC
• Mark your PSQL functions which do not have parameters
and return constant values with keyword DETERMINISTIC.
• The deterministic functions are calculated and cached in the
scope of the current query.

SQL OPTIMIZATION
Firebird 3.0
•Use analytical (window) functions
• If you are running SELECT with simultaneous output of some
column and aggregated function for it, use window (analytical)
functions
• it is faster than the subquery or 2 queries
SELECT id, department, salary, salary /
(select sum(salary) from employee) percentage
FROM employee
SELECT id, department, salary, salary /
sum(salary) OVER () percentage
FROM employee

SQL OPTIMIZATION
•Use switch -se for gbak
• Use switch –se to increase gbak backup and/or restore
speed up to 20%
gbak -b -g -se service_mgr c:dbdata.fdb e:backupdata.fbk

SQL OPTIMIZATION
•WHERE CURRENT OF
• The fastest way to process records fetched by the cursor in
PSQL is the clause
where current of <>
It is faster than
where rb$db_key = :v_db_key
and much faster than search with a primary or unique key.

SQL OPTIMIZATION
•Avoid often queries to monitoring tables
• Don't run queries to Firebird monitoring tables (MON$)
too often
such queries consume significant resources and can greatly
decrease the performance of the main business logic.
Recommend running MON$ queries not often than once
per minute.

SQL OPTIMIZATION
•Use NO_AUTO_UNDO option for bulk
inserts/updates
• If you are running many DML (Update/Insert/Delete)
commands in the frames of the same transaction, Firebird
merges undo-log of each command with undo-log of the
transaction.
• To speed up bulk DML operations start the transaction
with «NO AUTO UNDO» option, in order to do not merge
undo-logs of each command with the transaction's undo-
log.

SQL OPTIMIZATION
Firebird 3.0
•Do not use SRP authentication in if you
don't need it
• Does not use SRP users authentication if you don't really
need it
• Connect with SRP authentication is established slower
than the regular connection.

Thanks…
Fabio Codebue
Firebird Foundation
Committe Member
f.codebue@p-soft.biz
www.p-soft.biz
Firebird Optimization www.ibsurgeon.it

45 ways to speed up firebird database

Recommended

More Related Content

What's hot (20)

Similar to 45 ways to speed up firebird database (20)

Recently uploaded (20)

45 ways to speed up firebird database