Skip to main content

Dramatic performance boost using TEXT_IO instead of CLIENT_TEXT_IO

Last week we encoutered the problem that reading a 10,000 record csv-file took 45 minutes from a location in Belgium. From The Netherlands it took about 20 minutes. The Application Server and Database Server is located in the UK. The difference is due to available band width.
The performance from both locations were not acceptable. How did we solve this problem?

First we stripped down the code so that only the lines in the csv-file were read and no validation and processing (insert into table) took place. From Belgium only this took even 17 minutes!
Then we changed the code so that the file got transferred to the Application Server (using webutil_file_transfer) and replaced the use of the webutil package client_text_io with 'regular' text_io.
The results were astonishing: The transfer of the file is done in a split second and just reading the file line-by-line took less than a minute! After adding the validation and processing the process could finish in just 2 minutes.

So a little advice here:
only use client_text_io when you're reading a very small file, otherwise use webutil_file_transfer and text_io!
Post a Comment

Popular posts from this blog

Showing a success message after closing a modal dialog

APEX 5 comes with Modal Dialogs out of the box. Very neat. Especially for adding and changing data. And to minimise the number of time a user has to click, it could be useful to add a "Close Dialog" process after the actual data processing. When the data processing fails, the Dialog stays on top showing the error. When data processing runs fine, the Dialog is closed ... without any confirmation. And this might be scary for a shaky user.

So how can we provide the user some feedback? On Page 4 of the Sample Dialog Application you can see one solution: up on a Dialog Closed Event on the parent page it does a redirect to refresh the parent page appending the success message of the "Close Dialog" process. This has two drawbacks. First, it probably refreshes more than necessary. And second, if you're using multiple layers of dialogs (dialogs that open other dialogs) the message appears in the "parent dialog".
As an alternative you could follow these steps: 1…

APEX 5 New Static File Features

In APEX 4 you could upload files - like CSS files, JavaScript files, Images and whatever else you like - into the APEX Repository. When you navigate to Shared Components, there is a Files section that offers three different options:
CSS Files are always uploaded (and changed !) for the whole Workspace. For Images and Static Files (usually JavaScript) you could choose whether they should be available for the whole Workspace or for a specific Application only. And if you had a lot of files - e.g. a lot of images - then you had to go through the upload process one-by-one. But that's usually a one time only thing. If you make changes to the CSS and JavaScript files - and that's a continuous process in development - then you had to delete the existing file and upload the new one. Over and over again. And meanwhile fighting the cache of the webserver and your browser.  And another irritating issue: You couldn't use relative references in your CSS or JavaScript files as they just…

Using LDAP for Authentication and Authorization within APEX

One of my current customers would like to use their LDAP (Microsoft Active Directory) server for authentication and authorization of APEX applications. Of course we tried to set up a standard LDAP Authenication that's available within APEX. But we couldn't get that to work. Maybe it has to do with the fact that the client stored their Users within Groups within Groups within .... . Or maybe it doesn't do a full tree walk in the directory. Or maybe it is just because it is Microsoft - and not Oracle Internet Directory (OID). So we moved to a custom Authentication using the DBMS_LDAP functions (and some examples from the Pro Oracle Application Express book and Tim Hall - a.k.a. Oracle Base).

One of the issues we encountered that we wanted to use the user's login name, like "jdoe" and not his full name ("John Doe"). And the login name is stored in the "sAMAccountName" attribute. But authenticating using just "jdoe" didn't work. …