Sunday, August 8, 2010

Proxy Authentication with Thoth Gateway

Christian Vind submitted an enhancement request for the Thoth Gateway to support Oracle proxy authentication by passing on the current Windows username to the database connection string.

The point of proxy authentication is that

  • The proxy user only has "create session" privileges but can't do much else.
  • The real user does not have "create session" privileges and cannot log on to the database without knowing the proxy user name and password (and that is only set on the web/application server).
  • The USER function returns the real user name, and all standard database auditing, roles, etc. work as usual.

As of version 1.3 of the Thoth Gateway, proxy authentication is now supported. Here is how it works:


IIS Setup

Set up the application (virtual directory) in IIS where the gateway runs with Integrated Windows Authentication, so that the CGI environment variable LOGON_USER will be populated with the client's Windows username. (If the user is using Internet Explorer to browse the site, his identity will be passed on to the web server/gateway automatically; if using another browser, then an explicit logon is required.)

Oracle Setup

Define an "application server user", ie the common user that connections will be established through:


-- Log on as DBA (SYS or SYSTEM) that has CREATE USER privilege.


create user appserver identified by eagle;



create user end_user identified by secret;
grant create session to end_user;
alter user end_user grant connect through appserver;




Now test the setup with SQL*Plus, by connecting with the "application server user", and then "becoming" the end user:



-- note we don't specify the end_user password, but still become that user


SQL> connect appserver[end_user]/eagle
Connected.
SQL> select user from dual;


USER
------------------------------
END_USER


SQL>



Note that since the point of this is to take advantage of existing Active Directory accounts, you probably want to create your users like this:


create user "your_domain\end_user" identified externally;
grant create session to "your_domain\end_user";
alter user "your_domain\end_user" grant connect through appserver;




Thoth Gateway Setup

In web.config, modify the DAD settings (the following example assumes a local Oracle XE installation):


param name="DatabaseConnectString" value="//127.0.0.1:1521/xe"
param name="DatabaseConnectStringAttributes" value="Enlist=false;Proxy User Id=appserver;Proxy Password=eagle;"
param name="DatabaseUserName" value="LOGON_USER"
param name="DatabasePassword" value=""


Notice the value "LOGON_USER" specified for the DatabaseUserName parameter. This is a reserved string that will be replaced with the actual value of the LOGON_USER value from the web request (ie. the user's Windows username, typically "domain\username"). You can also specify "LOGON_USER_NO_DOMAIN" to strip away the domain part of the user name -- what you use will depend on how you have set up your user accounts in Oracle.

Testing It

To test that everything works at this point, create a procedure similar to the following, and execute it via the gateway (don't forget to grant execute privileges on it to the end-user's account, and create a public synonym for it unless you prefix with the procedure owner's name in the URL).


procedure test_proxy_auth
as
begin


  htp.header(1, 'Proxy authentication');


  htp.ulistopen;
  htp.listitem ('USER = ' || user);
  htp.listitem ('Proxy user = ' || sys_context('userenv', 'proxy_user'));
  htp.listitem ('CGI LOGON_USER = ' || owa_util.get_cgi_env('LOGON_USER'));
  htp.ulistclose;


end test_proxy_auth;



If successful, the USER function should return the end-user's Windows username, and the Proxy User should display as "appserver".

Postscript: A little enigma

Actually, if you do as described above, you could possibly get this error when you try to run the procedure via the gateway:

ORA-1045: user %s lacks CREATE SESSION privilege; logon denied


At least, that's what I got . To get around it, I had to explicitly grant this to the "appserver" user:

grant create session to appserver;

The funny thing is that my example above, tested via SQL*Plus, shows that this works without the grant! But when attempting the same connection via ODP.NET, it gives the above error unless the grant is made.

And if I revoke the "create session" from the end_user, the above example doesn't work in SQL*Plus, because of the missing privilege. Which seems to contradict the purpose of proxying, as defined at the top of this blog post.

If anybody knows why SQL*Plus and ODP.NET show different behaviour here, please let me know.

Thoth Gateway version 1.3 available

There is a new version of the Thoth Gateway, a mod_plsql replacement for IIS, available for download. The latest version is 1.3.

It contains the following bug fixes and enhancements:



  • Bug Fix: Issue with parsing client IP address: Added exception handling to prevent error when parsing client IP address with invalid format.
  • Ignore additional request parameters: Certain tools and frameworks may dynamically add additional parameters to a request, which causes the corresponding PL/SQL call to fail, since these parameters are not defined in the procedure signature. As of this version, the gateway will now retry the call after dropping (ignoring) any parameters that cannot be found in the Oracle data dictionary for the procedure being called.
  • Support for Oracle proxy authentication (and Single Sign On) via dynamic username substitution: Oracle proxy authentication, combined with Integrated Windows Authentication in IIS, allows you to pass the end-user's identity from the client to the database session (so the function USER will return the end-user's Windows username, with no login required). This is useful in an intranet scenario where users are defined in an Active Directory domain and use Internet Explorer to access the PL/SQL web application.


For existing installations, simply overwrite the existing PLSQLGatewayModule.dll file in the "bin" folder with the latest version from the downloaded archive.

Thursday, July 15, 2010

Apex on Thoth Gateway and IIS 7

Several people have asked for instructions on how to run the Thoth Gateway (a mod_plsql replacement) on Microsoft Internet Information Server (IIS) 7.

I finally had the chance (and the time!) to test the configuration on a server running IIS 7. The biggest challenge was understanding the new administration console user interface for IIS 7; it was was slightly confusing for someone who is used to IIS 6.

After a bit of fiddling, I got the gateway up and running:



I have updated the installation instructions in the latest download package (version 1.2.1) with separate sections for IIS 6 and IIS 7. So download, unzip and read the instructions in the "doc" folder.

And leave a comment if you run into problems (or leave a comment if you managed to get it up and running using the instructions!).

Saturday, June 26, 2010

Replacing Apex? More like Find and Replace...

I was getting my daily dose of Apex blog posts when I noticed this advertisement:

I was curious, so I clicked the link and got the following page (http://www.wavemaker.com/solutions/oracleforms.html):


"Well, this is strange", I thought... it says "Oracle Forms" there in the URL, and in the illustration in the middle of the page, yet the advertisement and the page heading talks about replacing "APEX". There is also a claim that the latter "costs a fortune", but as we all know Application Express is a no-cost option in the Oracle database. WTF?

There is also a link to an 8-page whitepaper on "Migrating Oracle Apex Applications to Java" (http://www.wavemaker.com/pdf/Migrating-Oracle-Apex-Apps-To-Java-With-WaveMaker.pdf). This whitepaper includes the following screenshot:



As well as this table:


OK, so it is clearly Oracle Forms that is depicted and described here, but labeled as if it was Oracle Apex.

At best, this is a clueless mistake made by some marketing sod who did a global Find & Replace from FORMS to APEX. At worst, this is deliberately misleading.

In any case, it's just wrong, wrong, wrong. I don't know about you, but I would not trust a company that is either incompetent, dishonest, or both. I'll stick with Apex, thank you.

Tuesday, June 15, 2010

Small patch for the Thoth Gateway (Apex on IIS)

Just a quick note to announce that a minor patch release (version 1.2.1) of the Thoth Gateway is available for download. The Thoth Gateway allows you to run Oracle Apex applications on the Microsoft IIS web server (a replacement for Apache and mod_plsql).

The following has changed:
  • Bug Fix for Content-Length: Fixed an issue where the content-length header would be incorrectly set for non-AL32UTF8 databases if the page contained multibyte characters.
  • Set ODP.NET connection string attributes: Added option to specify additional connection string attributes in the DAD configuration. This allows you to fine-tune the connection properties. See the ODP.NET documentation for more details.
Upgrade for existing installations: Simply download the latest version, unzip the archive and drop the PLSQLGatewayModule.dll file into your bin folder on the web server.

Friday, May 7, 2010

ApexGen has a new home

My first open-source project was ApexGen, a utility to generate Oracle Application Express (Apex) pages from PL/SQL, in a fraction of the time it takes to create Apex pages manually (using point-and-click). I originally hosted it on SourceForge. However, since then I have started a few other projects hosted on Google Code, so I decided to move ApexGen to Google Code as well.

To summarize, here are my current Oracle, Apex and PL/SQL projects, all on Google Code:

  • Thoth Gateway, a mod_plsql replacement that runs on Microsoft Internet Information Server (IIS). It allows you to use IIS as the web server for Apex applications (instead of Apache or the Embedded PL/SQL Gateway), and it has a few extra features as well, such as CLOB support, automatic Web Services published from PL/SQL, XDB integration, and integrated Windows authentication out-of-the-box.
  • JQGrid Integration Kit for PL/SQL, a set of PL/SQL packages that allows you to use the JQGrid component to display and edit tabular data in your Apex applications. It is faster, better-looking and more flexible than the built-in tabular forms in Apex.
  • ApexGen, a utility to generate Oracle Application Express (Apex) pages from PL/SQL, in a fraction of the time it takes to create Apex pages manually. With Apex 4.0 just around the corner, I believe ApexGen will be due for an overhaul soon, as the export files are likely to have changed quite a bit.

Saturday, April 10, 2010

SELECT * FROM spreadsheet (or How to parse a CSV file using PL/SQL)



I recently needed to retrieve/download a comma-separated values (CSV) file from a website, and insert the data in an Oracle database table.

After googling around a bit, I found various pieces of the solution on AskTom, ExpertsExchange and other sites, which I put together in the following generic utility package for CSV files.

Usage


Because I have implemented the main parsing routine as a pipelined function, you can process the data either using straight SQL, or in a PL/SQL program.



For example, you can retrieve a download a CSV file as a clob directly from the web and return it as a table with a single statement:


select *
from table(csv_util_pkg.clob_to_csv(httpuritype('http://www.foo.example/bar.csv').getclob()))


And maybe do a direct insert via INSERT .. SELECT :

insert into my_table (first_column, second_column)
select c001, c002
from table(csv_util_pkg.clob_to_csv(httpuritype('http://www.foo.example/bar.csv').getclob()))


You can of course also use SQL to filter the results (although this may affect performance):

select *
from table(csv_util_pkg.clob_to_csv(httpuritype('http://www.foo.example/bar.csv').getclob()))
where c002 = 'Chevy'


Or you can do it in a more procedural fashion, like this:

create table x_dump
(clob_value clob,
 dump_date date default sysdate,
 dump_id number);


declare
  l_clob clob;

  cursor l_cursor
  is
  select csv.*
  from x_dump d, table(csv_util_pkg.clob_to_csv(d.clob_value)) csv
  where d.dump_id = 1;

begin

  l_clob := httpuritype('http://www.foo.example/bar.csv').getclob();
  insert into x_dump (clob_value, dump_id) values (l_clob, 1);
  commit;
  dbms_lob.freetemporary (l_clob);

  for l_rec in l_cursor loop
    dbms_output.put_line ('row ' || l_rec.line_number || ', col 1 = ' || l_rec.c001);
  end loop;

end;



Auxiliary functions


There are a few additional functions in the package that are not necessary for normal usage, but may be useful if you are doing any sort of lower-level CSV parsing. The csv_to_array function operates on a single CSV-encoded line (so to use this you would have to split the CSV lines yourself first, and feed them one by one to this function):

declare
  l_array t_str_array;
  l_val varchar2(4000);
begin

  l_array := csv_util_pkg.csv_to_array ('10,SMITH,CLERK,"1200,50"');

  for i in l_array.first .. l_array.last loop
    dbms_output.put_line('value ' || i || ' = ' || l_array(i));
  end loop;

  -- should output SMITH
  l_val := csv_util_pkg.get_array_value(l_array, 2);
  dbms_output.put_line('value = ' || l_val);

  -- should give an error message stating that there is no column called DEPTNO because the array does not contain seven elements
  -- leave the column name out to fail silently and return NULL instead of raising exception
  l_val := csv_util_pkg.get_array_value(l_array, 7, 'DEPTNO');
  dbms_output.put_line('value = ' || l_val);

end;



Installation


In order to compile the package, you will need these SQL types in your schema:

create type t_str_array as table of varchar2(4000);
/

create type t_csv_line as object (
  line_number  number,
  line_raw     varchar2(4000),
  c001         varchar2(4000),
  c002         varchar2(4000),
  c003         varchar2(4000),
  c004         varchar2(4000),
  c005         varchar2(4000),
  c006         varchar2(4000),
  c007         varchar2(4000),
  c008         varchar2(4000),
  c009         varchar2(4000),
  c010         varchar2(4000),
  c011         varchar2(4000),
  c012         varchar2(4000),
  c013         varchar2(4000),
  c014         varchar2(4000),
  c015         varchar2(4000),
  c016         varchar2(4000),
  c017         varchar2(4000),
  c018         varchar2(4000),
  c019         varchar2(4000),
  c020         varchar2(4000)
);
/

create type t_csv_tab as table of t_csv_line;
/


UPDATE 04.04.2012: The latest version of the package itself (CSV_UTIL_PKG) can be found as part of the Alexandria Utility Library for PL/SQL.


Performance


On my test server (not my laptop), it takes about 35 seconds to process 12,000 rows in CSV format. I don't consider this super-fast, but probably fast enough for many CSV processing scenarios.

If you have any performance-enhancing tips, do let me know!

Bonus: Exporting CSV data


You can also use this package to export CSV data, for example by using a query like this.

select csv_util_pkg.array_to_csv (t_str_array(company_id, company_name, company_type)) as the_csv_data
from company
order by company_name

THE_CSV_DATA
--------------------------------
260,Acorn Oil & Gas,EXT
261,Altinex,EXT
262,Amerada Hess,EXT
263,Atlantic Petroleum,EXT
264,Beryl,EXT
265,BG,EXT
266,Bow Valley Energy,EXT
267,BP,EXT

This might come in handy, even in these days of XML and JSON ... :-)