Dramatic performance boost using TEXT_IO instead of CLIENT_TEXT

Dramatic performance boost using TEXT_IO instead of CLIENT_TEXT_IO

Last week we encoutered the problem that reading a 10,000 record csv-file took 45 minutes from a location in Belgium. From The Netherlands it took about 20 minutes. The Application Server and Database Server is located in the UK. The difference is due to available band width.
The performance from both locations were not acceptable. How did we solve this problem?

First we stripped down the code so that only the lines in the csv-file were read and no validation and processing (insert into table) took place. From Belgium only this took even 17 minutes!
Then we changed the code so that the file got transferred to the Application Server (using webutil_file_transfer) and replaced the use of the webutil package client_text_io with 'regular' text_io.
The results were astonishing: The transfer of the file is done in a split second and just reading the file line-by-line took less than a minute! After adding the validation and processing the process could finish in just 2 minutes.

So a little advice here:
only use client_text_io when you're reading a very small file, otherwise use webutil_file_transfer and text_io!

Comments

apex_application.g_f0x array processing in Oracle 12

If you created your own "updatable reports" or your custom version of tabular forms in Oracle Application Express, you'll end up with a query that looks similar to this one: then you disable the " Escape special characters " property and the result is an updatable multirecord form. That was easy, right? But now we need to process the changes in the Ename column when the form is submitted, but only if the checkbox is checked. All the columns are submitted as separated arrays, named apex_application.g_f0x - where the "x" is the value of the "p_idx" parameter you specified in the apex_item calls. So we have apex_application.g_f01, g_f02 and g_f03. But then you discover APEX has the oddity that the "checkbox" array only contains values for the checked rows. Thus if you just check "Jones", the length of g_f02 is 1 and it contains only the empno of Jones - while the other two arrays will contain all (14) rows. So for ...

Filtering in the APEX Interactive Grid

Remember Oracle Forms? One of the nice features of Forms was the use of GLOBAL items. More or less comparable to Application Items in APEX. These GLOBALS where often used to pre-query data. For example you queried Employee 200 in Form A, then opened Form B and on opening that Form the Employee field is filled with that (GLOBAL) value of 200 and the query was executed. So without additional keys strokes or entering data, when switching to another Form a user would immediately see the data in the same context. And they loved that. In APEX you can create a similar experience using Application Items (or an Item on the Global Page) for Classic Reports (by setting a Default Value to a Search Item) and Interactive Reports (using the APEX_IR.ADD_FILTER procedure). But what about the Interactive Grid? There is no APEX_IG package ... so the first thing we have to figure out is how can we set a filter programmatically? Start with creating an Interactive Grid based upon the good o...

Stop using validations for checking constraints !

If you run your APEX application - like a Form based on the EMP table - and test if you can change the value of Department to something else then the standard values of 10, 20, 30 or 40, you'll get a nice error message like this: But it isn't really nice, is it? So what do a lot of developers do? They create a validation (just) in order to show a nicer, better worded, error message like "This is not a valid department". And what you then just did is writing code twice : Once in the database as a (foreign key) check constraint and once as a sql statement in your validation. And we all know : writing code twice is usually not a good idea - and executing the same query twice is not enhancing your performance! So how can we transform that ugly error message into something nice? By combining two APEX features: the Error Handling Function and the Text Messages! Start with copying the example of an Error Handling Function from the APEX documentation. Create this function ...

Roels Blog

Search This Blog