Ad
More Related Content
What's hot (8)
Viewers also liked (6)
Ad
Similar to Firebird's Big Databases (in English) (20)
Ad
More from Alexey Kovyazin (20)
Recently uploaded (20)
Firebird's Big Databases (in English)
- 1. Firebird’s big databasesDmitry Kuzmenko,[email protected]
- 2. AgendaWhat is a big Firebird database?1 Terabyte (1024Gb) database testReal-world examplesOur statistics for databases sizes growthPeculiarities of big databases maintenanceBackup/restore approachHealth checks for indices and dataAutomated maintenance with FBDataGuardLicensing changes in IBSurgeonNo more “Personal”Focus on preventing corruptions, not fixing
- 3. What is a big Firebird database?
- 4. What is a big Firebird database? Pre-history2005 IBSurgeon found 37 gigabytes per table limit in Firebird 1.0-1.5 and all InterBase version2006Firebird 2.0 fixes that limit, and introduces nbackupAVARDA (ERP) shows working with the 100gb database2009 – IBSurgeon created 1 TB Firebird databasehttp://www.ib-aid.com/articles/item104
- 5. What is a big Firebird database? Examples from our customersBAS-X, Australia, warehouse & ERP200-450 GB, no BLOBsWatermark, UK, scanned documents and full-text search300+ GBs, BLOBsProfitmed, Russia, medical distribution company50 GB, no BLOBs Avarda, Russia, ERP100Gb, no BLOBsMegafon, Russian cell operator50 GB, cell information storage, no BLOBs
- 6. Sizes of Firebird databases (IBSurgeon’s customers statistics)
- 7. Peculiarities of big databases maintenance
- 8. Usually Firebird database draws simplyFirebirdbecause it is believed as something simple.
- 9. Zero maintenance?Self-tuned in 90% casesDefault firebird.conf is good enoughDefault database parameters: sweep, buffers, etc are good enoughPage size 4096 is OkMaintenance is simpleJust make backup/restoreNot true for big databases.
- 10. Big databases are hard to maintainBackup/restore cycle can take hoursDepends on data density and HDD performance – numeric data are restored slowlyBad example - 1 hour to restore 3.6 GB databaseGood example – 12 hours to backup 450 Gb databaseSweep takes from 30 minutes to 4 hoursFor 50Gb databasegfix of the 37 GB database takes 1-2 hoursRestore of 50 GB database takes 4 hours
- 11. Big database has high business requirementsProfitmed~450 concurrent users50 gigabyte database with very intensive load, +1Gb/month12 hours x 7 days work modehigh availability requirements"warn us at least 5 minutes before crush“Losses in case of 1 day stop ~$50000 Firebird 1.5 Classic 64 bit64-bit CentOS Linux, 64Gb RAM, SCSI Single-user backup takes 30 minutes
- 12. Multi-user backup takes 2 hours
- 13. Restore takes 4 hours
- 14. nbackup is not available in Firebird 1.5, plain copy is used (with shutdown)
- 15. Sweep
- 16. only manual
- 17. from 30 minutes to 1 hour
- 18. 2 times per dayHow to deal with such big database and fulfill heavy business requirements?We need to look into details of Firebird database
- 19. Firebird detailsFiles structuredelta- Main database file0-level- Volume filesDatabase file- delta (nbackup) andincremental backupsVolume 1Volume N- External tables
- 20. Firebird detailsInternal structureMetadataData in tablesIndicesBLOBs
- 21. Key difference between small and big databasesSmall database – backup/restore and relaxJust make backup/restore – it creates database from scratch and removes all problemsTransaction issues and other problems does have time to grow between backup/restore cyclesBig database – need to check and maintain in several steps…
- 22. Big database requires individual maintenance plan Maintenance plan depends on size of database and work mode (8x5, 24x7)Backups scheme is not simplePerform test restores separatelyTo be checkedErrors – in firebird.log and run error checking quries on live databaseMetadata – check integrity Metadata limitsData & BLOBs – walk through data, check segmentationIndices – check indices healthTransactions – any gaps, garbage growth, other problems
- 23. Everyday minimal (!) maintenance plan for big database
- 24. Example of backup plan for big Firebird databaseMaintenance serverMain serverFirebird databaseNbackup copyChecking restoreGbak-bAnd each step should be confirmed and reported.
- 25. What to monitor-1Server and database errorsServer and database availabilityCheck all changes in firebird.logCheck metadataTransactionsTransaction markers monitoring (garbage problems)Limit (2 billions between backup/restore)UsersMin/max/avg users
- 26. What to monitor-2Database filesSingle volume and multi-volumePaths – where to stored (not at the same drive with temp files and backups!)Sizes and growth limitsDelta-files (nbackup)Life-time and sizesBackup filesExistence, sizes and growth limits
- 27. What to monitor-3Temp files FirebirdOverall size and quantityNumber of formats per tableNo more than 255Less formats in productionNon-activated and deactivated indicesDeactivated – explicitly deactivated (why deactivated?)Non-activated – indicates problems during restore
- 28. What to monitor-4Periodical statistics (gstat)Firebird server versionLatest patches are recommendedFirebird installation sizeFirebird logs size and paths
- 29. Maintenance-1BackupsRevolver (days, week, month copies) backupsBackup depthChecking restore (need to check results)Growth prognosis (if not enough space, backup should be canceled)Control backup time (too long backup indicates problems)
- 30. Maintenance-2IndicesRecalculate indices statisticsSelected or excludedCheck index status – active/in-active/non-activatedCheck physical index health
- 31. Maintenance-3Validate database with gfixDon’t forget to shutdown databaseAnalysis (includingfirebird.log)Metadata validationCheck important system tablesFirebird.log maintenanceWhen log becomes very big, copy it to backup log filesAnd some more things….
- 32. Your administrator definitely does all these steps…
- 33. …especially in remote offices.
- 34. And this is not enough!Business wants to have warranty - even if hardware fails data should be recovered!
- 35. That’s why we created FBDATAguard
- 36. FBDataGuarddoes all above things…Watches database files, volumes, deltas, performs and checks backups in the right wayVerifies metadata, data and indicesWatches for errors, limits and wrong versionsSends alerts and recommendations
- 37. And even more – protects from hardware failuresFBDataGuard Extractor extracts data from corrupted database and inserts to the newMetadata repositoryNew БДTables dataBLOBs
- 38. Firebird DataGuardWatch for 26 important database and server parametersAlerts for potential and real problem by emailProper automation of database maintenanceWindows, Linux, MacOS, Firebird 1.5-2.1Special licensing for ISV (Independent Software Vendors) Firebird developers
- 40. Why to change?Prevent corruptions instead of fixingIt’s time to fight reasons, not consequencesWwronglicense understandingNo more PersonalSoftware-As-A-Service offer – 1 month and 1 instance
- 41. Upgrade pathsPack Support 4.0FirstAID Tech Support 3.0Cancelled
- 42. Only at FDD 2010