Skip to main content

Dramatic performance boost using TEXT_IO instead of CLIENT_TEXT_IO

Last week we encoutered the problem that reading a 10,000 record csv-file took 45 minutes from a location in Belgium. From The Netherlands it took about 20 minutes. The Application Server and Database Server is located in the UK. The difference is due to available band width.
The performance from both locations were not acceptable. How did we solve this problem?

First we stripped down the code so that only the lines in the csv-file were read and no validation and processing (insert into table) took place. From Belgium only this took even 17 minutes!
Then we changed the code so that the file got transferred to the Application Server (using webutil_file_transfer) and replaced the use of the webutil package client_text_io with 'regular' text_io.
The results were astonishing: The transfer of the file is done in a split second and just reading the file line-by-line took less than a minute! After adding the validation and processing the process could finish in just 2 minutes.

So a little advice here:
only use client_text_io when you're reading a very small file, otherwise use webutil_file_transfer and text_io!
Post a Comment

Popular posts from this blog

Push changed rows to an Interactive Grid

For pushing changes from the database to the end user, the regular solution is using websockets. A change in a record is detected - using a trigger or using the CQN (Change Query Notification) feature - and a notification is send to a websocket server. That websocket server broadcasts the notification over a channel to all browsers that are tuned in to that websocket channel. Then the browser reacts to that notification, usually showing an alert or refreshing a report. This trick is described on multiple sites, just Google for "oracle apex websockets" or similar.

So back in the old days, we used that notification in the browser to refresh the (interactive) report. But along comes the Interactive Grid (IG). While he full-refresh mechanism still works for IG, an IG has also the option to refresh just one row.  So wouldn't it be awesome that just the changed row(s) get refreshed upon a change in the database, instead of the whole report? Can we do it ... yes we can!
First i…

A review of APEX World 2017 - Day 1

Last week the SS Rotterdam was the beautiful location of the largest gathering of APEX Developers worldwide. With around 380 (!) attendees a new high was set. And they came from all over the world : I spotted people from The Netherlands, Belgium, Switzerland, Austria, Croatia, Germany, Denmark, Norway, UK, Ireland and the USA. And I even might have missed one or two ….

The event started with a presentation by the “father of APEX”, Mike Hichwa, talking about "Oracle APEX Past, Present and Future”. Of course everyone is curious what the APEX future might bring: Friendly URL’s, automated testing, more JSON, concurrent APEX versions, third party Oauth 2 authentication (think Facebook, Google), APEX app diff and more, a lot more, REST capabilities. And now we have to wait for APEX 5.2 … and that might take a while! 
After this keynote, the conference split up in three tracks. After the coffee break I returned to to big theatre where Geertjan Wielenga talked about "Finally Javas…

Dockerize your APEX development environment

Nowadays Docker is everywhere. It is one of the main components of Continuous Integration / Continuous Development environments. That alone indicates Docker has to be seen more as a Software Delivery Platform than as a replacement of a virtual machine.

However ...

If you are running an Oracle database using Docker on your local machine to develop some APEX application, you will probably not move that container is a whole to test and production environments. Because in that case you would not only deliver a new APEX application to the production environment - which is a good thing - but also overwrite the data in production with the data from your development environment. And that won't make your users very excited.
So in this set up you will be using Docker as a replacement of a Virtual Machine and not as a Delivery Platform.
And that's exactly the way Martin is using it as he described in this recent blog post. It is an ideal way to get up and running with an Oracle database …