Skip to main content

Dramatic performance boost using TEXT_IO instead of CLIENT_TEXT_IO

Last week we encoutered the problem that reading a 10,000 record csv-file took 45 minutes from a location in Belgium. From The Netherlands it took about 20 minutes. The Application Server and Database Server is located in the UK. The difference is due to available band width.
The performance from both locations were not acceptable. How did we solve this problem?

First we stripped down the code so that only the lines in the csv-file were read and no validation and processing (insert into table) took place. From Belgium only this took even 17 minutes!
Then we changed the code so that the file got transferred to the Application Server (using webutil_file_transfer) and replaced the use of the webutil package client_text_io with 'regular' text_io.
The results were astonishing: The transfer of the file is done in a split second and just reading the file line-by-line took less than a minute! After adding the validation and processing the process could finish in just 2 minutes.

So a little advice here:
only use client_text_io when you're reading a very small file, otherwise use webutil_file_transfer and text_io!
Post a Comment

Popular posts from this blog

Showing a success message after closing a modal dialog

APEX 5 comes with Modal Dialogs out of the box. Very neat. Especially for adding and changing data. And to minimise the number of time a user has to click, it could be useful to add a "Close Dialog" process after the actual data processing. When the data processing fails, the Dialog stays on top showing the error. When data processing runs fine, the Dialog is closed ... without any confirmation. And this might be scary for a shaky user.

So how can we provide the user some feedback? On Page 4 of the Sample Dialog Application you can see one solution: up on a Dialog Closed Event on the parent page it does a redirect to refresh the parent page appending the success message of the "Close Dialog" process. This has two drawbacks. First, it probably refreshes more than necessary. And second, if you're using multiple layers of dialogs (dialogs that open other dialogs) the message appears in the "parent dialog".
As an alternative you could follow these steps: 1…

It's happening again ... running for the ODTUG Board of Directors 😉

For the third time in a row I'll be running for ODTUG's Board of Directors. But after ending as a runner up twice, I am sure I'm going to make it this time! But not without your help!

My campaign statement this year is:
I have been attending and presenting at Kscope conferences since 2007. This not only resulted in a vast amount of knowledge, but also - and even more important - a huge number of friends from all over the globe.  I want to see ODTUG grow and spread this community feeling even more! 
My experience as an attendee, presenter and content lead has provided the basic foundation to be a director. Next to that, my personality and (global) network will be beneficial to the whole board and organization. 
Since March I have served on the Board of Directors in a limited term for a Director who stepped down due to a career change. This has allowed me to have unique insight of all the things that are going on in and around the ODTUG organization. As the train was already ro…

A review of APEX World 2017 - Day 1

Last week the SS Rotterdam was the beautiful location of the largest gathering of APEX Developers worldwide. With around 380 (!) attendees a new high was set. And they came from all over the world : I spotted people from The Netherlands, Belgium, Switzerland, Austria, Croatia, Germany, Denmark, Norway, UK, Ireland and the USA. And I even might have missed one or two ….

The event started with a presentation by the “father of APEX”, Mike Hichwa, talking about "Oracle APEX Past, Present and Future”. Of course everyone is curious what the APEX future might bring: Friendly URL’s, automated testing, more JSON, concurrent APEX versions, third party Oauth 2 authentication (think Facebook, Google), APEX app diff and more, a lot more, REST capabilities. And now we have to wait for APEX 5.2 … and that might take a while! 
After this keynote, the conference split up in three tracks. After the coffee break I returned to to big theatre where Geertjan Wielenga talked about "Finally Javas…