Oracle InMemory hardcore edition

Editor's Notes

• #2: Hello, guys. My name is Alex and I work for DataArt as a performance architect. Does someone use Oracle database in your applications? Have you ever used Oracle InMemory option to bring a performance profit? Perfect. Today we are going to discuss how we improved faceted search performance with Oracle InMemory option for one of our client. Важно, что я не буду влезать в технические детали реализации отдельных фишек, а просто расскажу, почему у нас они не взлетели и как мы их заводили, иначе наш доклад будет очень долгим
• #3: Let me introduce myself I work for DataArt as a performance architect and my main focus is data warehousing for big clients. I have huge Oracle experience but I works with Exasol and Tarantool databases either.
• #4: We have rather straightforward agenda for today discussion We’ll discuss what is faceted search, client application overview and its performance issues I try to explain how Oracle InMemory option is implemented internally but without digging into details. My main focus is how we made the customer happy and share with you our issues you could face with as well For sure I’ll answer your questions.
• #5: Everybody know what is faceted search. We have groups named facets, Values named constraints and Figures names facet counts
• #6: Common facet types are terms Intervals And ranges
• #7: We need facets if Users need to filter content using multiple taxonomy terms at the same time. Users want to combine text searches, taxonomy term filtering, and other search criteria. trying to discover relationships or trends between contents. Navigate inside tons of data And your end users are already frightened by "advanced" search forms in enterprise applications
• #8: There are many ways how to implement faceted search internally and one of them is using tags. Everyone knows what is tags but let me reming. Tag is a keyword assigned to an object Mostly is is chosen informaly If it is assigned by the viewer and tags count is unlimited it is folksonomy When assigned by the creator and a set of tags is limited it is taxonomy
• #9: So we could store data for facets in tags which is extremely convenient for terms faceting by folksonomies and taxonomies Or in a plain table which is very convenient for range, interval facets defined by taxonomies. I think everything is clear enough but may be you have some questions about theory? So, let’s move on
• #10: We use taxonomy and foksonomy, so the best part of our facets are terms and we choosen tag-based storage.
• #11: Let me show some statistics. We extract objects, tags, facets for 2 years We have about 3 000 000 objects tagged, 42 000 000 terms applied, 100 000 unique terms, maximum tags count is 15 and there are about 150 facets. Is it whether decent volume or not? I think it is decent because we are bigger than StackOverflow by tags in 3 times.
• #12: We have ordinal architecture for enterprise. Web browser, balancer, a set of application servers which deal with inmemory grid and oracle with the replica on the backend. Pay attention that there is no full text search server which has been prohibited in accordance with customer policies.
• #13: We have microservice architecture and one of the services it tagging service which is responsible for data ingestion and faceted search. It is widely used by many other services and receives about 10000 requests per second. In order to serve search requests there are about 20 fine-tuned sql queries.
• #14: I can’t show you real printscreens from the application because of NDA so I implemented the mockup in Paint. As we could see UI is full of features as Predefined faceted search templates Full text search by random tags. Here it is done by names. We could search via audit information and we have faceted navigation for sure. We could filter out objects where some tags are present (not tag values) and user paging. It is rather sophisticated UI.
• #15: Without digging into details I could say that the best part of search is made on a wide Denormalized table with tags which stores only current tag values
• #16: We could have a look into the table in details. It contains tagged object types, facet information, tags information and values. There is Denormalized object name and object description tags because they are used in full text search mostly.
• #17: Let’s have a look into performance metrics. Pay attention the insert performance is very good so I used logarithmic scale to put everything in one diagram. On a first glance figures look good. We do complicated searches less than zero point two seconds but actually it is bad because we need to serve more and more users and data so queries become slow
• #18: In accordance with one of my otherspresentations the best option for faceted search is full text search dedicated server but we had a lack of time to implement it as well as reluctant to introduce additional layers
• #19: We started looking into InMemory solutions especialy regarding the customer already had Oracle InMemory license. In accordance with Oracle documentation it should be used for analytical queries which isn’t our case completely but we decided to try.
• #20: Why we should try Because we are under limitations preasure The customer has a license The table is rather wide, full of data which has a lot of equal values So it will fit in memory especialy regarding compression features And last but not least queries are full of aggregate functions
• #21: In accordance with the Oracle documentation InMemory feature is sort of silver bullets so we decided not waste time understanding how it works internaly. What we knew that the option stores row and columnar data in dual mode maintaining their consistency automatically.
• #22: Let me remind you our table
• #23: We set InMemory area size to 10 gigabytes and assigned the table into the area. We decided that performance will be faster without compression so no compression option was used.
• #24: What is significant to stress that InMemory brought some overhead without compression
• #25: So we started our tests and faced with the difference in performance was extremely unsignificant which is expect for the search by primary keys but not for search by tags
• #26: So we started our tests and faced with the difference in performance was extremely unsignificant which is expect for the search by primary keys but not for search by tags
• #27: So we started our tests and faced with the difference in performance was extremely unsignificant which is expect for the search by primary keys but not for search by tags
• #28: When we started our POC we relied on Lufthansa usecase when they got average 21 performance profit but our profit was 10 times less. It means that we do something wrong and we need to understand how InMemory works internaly.
• #29: Do you recall I mentioned InMemory area? It consists from 2 parts actualy. The first one in IMCU – InMemory Compression Unit which stores data in columnar format And SMU – Snapshot Metadata Unit which stores transaction related metadata, dictionaries and Zone Maps. Do you know what is zone map?
• #30: Let’s have a look into simplified example. We have sales table which is split by 1 Mb IMCU blocks. Each block stores minimal and maximum values for each column so when we need to search sales for California we scans zone maps and check does our value is located in min/max boundaries so we do not scan extra blocks.
• #31: I converted SMU zone maps in readable format. As you could see they are stored per-column base as I told before. You see numeric zone maps and character ones. Please pay attention that dictionary_entries column contains only zero.
• #32: As we understood it happens because no compression were switched on.
• #33: Let’s enable the compression
• #34: We’ve seen that dictionary extries had been populated which should give dictionary-based pruning in accordance with the documentation Кстати, не все zone maps одинаково полезны. Как выдумаете какая из них будет абсолютно бесполезна и не будет использоватьс?
• #35: We’ve seen that dictionary extries had been populated which should give dictionary-based pruning in accordance with the documentation Не работает оно чаще всего из-за того, что zone maps дают слишком широкий диапазон. Более того, нет статистик, которые позволяют отличать причину CU pruning – dictionary based или Zone maps
• #36: Compression ration is rather high but it depends from you data for sure. You could see that compression doesn’t influence on ingestion so high compression could be used as the default. Frankly speaking loading 6 gigabytes for 1 minute it is to much so we decided to have a look into FastStart feature.
• #37: The feature writes InMemory area in a columnar format and once database is rebooted it loads the data in memory with the speed of file reading so we got 3 times loading profit.
• #38: Тем не менее вчера я увеличил коэффициент pruning и иногда компрессию использовав как вы думаете что? Самое обидное, что до этого слайда я дошёл только вчера и проверил его. Само собой я стал гуглить, чтобы убедиться., что я уникален в своём открытии, но оказалось,, что Мария Колган тоже нашла решение с order by, но решение с clustering это моё секретное ноухау.
• #39: It is time for rerun. Try to guess which performance profit we got once we used high compression? Actually no profit at all. Поэтому придётся разбираться в деталях
• #40: В Oracle InMemory очень много умных трансформаций, которые видно в плане и чем более сложные джоины и агрегаты тем более вероятно их увидеть. Одной из них является InMemory aggregation, которая показывается такой умной картинкой. Ключевое для нас в ней то, что создаётся аналог bloom фильтра Key vector и по нему происходит джоин sales с stores и products и при этом считаются суммы в процессе скана sales.
• #41: Помните, я показывал в прошлой презентации sql для поиска по нескольким условиям с хитрыми групп бай. Именно для них использовался механизм vector transformation, который задуман для inmemory агрегаций, но как показал наш опыт серьёзно ускоряет джоины на запросах с агрегатами. Так вот всё это на полях типа binary, которые были в джоинах заменялось на блум фильтр, который был медленнее.
• #42: Мы попробовали решить эту задачу через так называемые JOING GROUPS. Это аналоги JOIN CLUSTER
• #43: Сделали join group. Кстати, вопрос. Как понять по плану или по статистикам, что join group используются? А никак! ни в плане ни в статистиках её не видно. Её видно только в Realtime SQL monitor, поэтому пришлось парсить его XML и через него вычислять. Стали разбираться когда они используются и результат такой. Они используются только тогда, когда в условие джоина есть поля, указанные в join groups, а по нашим запросам это было не всегда возможно.
• #44: Changed all types to bigint and removed indexes
• #45: We understood that no reasons to store all columns inmemory because we search by them via Oracle Full Text Search indexes
• #46: So the final structure is simple data types without b-tree indexes and 2 fields are removed from inmemory storage
• #47: We did some performance tests and found sporadic search degradation which happens where records are inserted Помните, я говорил, что у меня нет презентаций без звоночков!
• #48: Let’s understand how Oracle provides read consistency for InMemory Once a record is changed it is marked as stale. All stale records are read either from buffer cache or disk. Oracle repopulates stale records by staleness threshold or each 2 minutes by so named trickle repopulation process. It could be done in parallel plus we could state how much resources could be consumed for stale records repopulation
• #49: Собственно я думаю эти параметры отвечают за инвалидацию IMCU, но у нас не хватало времени изучить это детально. На самом деле их около 34, но трогать их весьма боязно.
• #50: Как же можно понять, что происходит чтение с диска, а не с InMemory? Очень просто Эти две статистики должны быть по нулям
• #51: За самим процессом пересчёта IMCU можно наблюдать через 2 представления
• #52: Тут пойман очень интересный процесс. Начался процесс trickle_repopulation, было обнаружено. Что затронуто очень много данных и произошла полная перезагрузка
• #53: Let’s have a look what we have by default
• #54: As soon as we set 4 repopulation servers and 8 percent of time to repopulate stale records and performance spikes disappeared
• #55: InMemory option is very easily to trace and maintain but let me share a secret information with you. We needn’t to know all of them at all.
• #56: 8 parameters covers the best part of InMemory performance tuning. Let me share one more secret. only 1 parameter is enough. Try to guess which one. It states how efficient zone maps indexes are used.
• #57: 3 settings covers the best part of dba activities
• #58: And 2 views give us enough information for basic InMemory undertanding
• #59: Let’s have a look into final figures. As you could see we have about 4-5 times performance profit which isn’t so good but regarding our use case is fine.
• #60: Let’s have a look into final figures. As you could see we have about 4-5 times performance profit which isn’t so good but regarding our use case is fine.
• #61: Let’s have a look into final figures. As you could see we have about 4-5 times performance profit which isn’t so good but regarding our use case is fine.
• #62: Let’s elaborate DBA findings. InMemory area size should be estimated taking into account compression and your data If we have all data in memory it doesn’t mean we are extremely fast InMemory takes time to be loaded after server reboot 8 statistics is enough to help developers with the best part of their issues Alling tackle processes settings with your workload
• #63: Findings for developers are that in case of our usecase extremely advanced inmemory features like join groups, inmemory aggregations hasn’t brought significant performance profit. Zone maps work perfect with numeric and date data types. Dictionary based pruning doesn’t work in our oracle version at least. More simple data types are used – more performance we have And extremely significant that high compression doesn’t affect ingestion speed То, что происходит repopulate даже в случае незакомиченной транзакции радикально отличает от result cache.
• #64: Faststart Add some star schema data with star – try what for IM is intended initialy Tell about cool stuff with multithread scans
• #65: Our main conclusion is that you shouldn’t rely on documentation and experts – try and measure your particular use-case IM works with short queries as well Without understanding how it works you will not get performance profit You have to change application to use simple data types What is good no extra layers, message queues, hardware need to be added We had managed to put our IM POC in production very fast (1 week actualy) And business got 5 times performance boost. Probably that’s it from my side for today.
• #66: Thank you for your time and I’m ready answer your questions