I was looking at the Apex 4.2.2 installation script (coreins.sql), and noticed the mention of a new package, wwv_flow_json, which is apparently being worked on but was removed at the last minute "as no longer required for 4.2.2".
Even though the package is not installed into the database in Apex 4.2.2, the source code files are still present in the download. Looking at wwv_flow_json.sql we can see that this is a "JSON lexer and parser":
Now, obviously there is no guarantee of this package appearing in the next Apex release (5.0?), and the package spec even says it is not a public package and not for runtime deployment.
However, I would love to see an APEX_JSON package (that is, a synonym to the wwv_flow_json package) being bundled with Apex as an official, supported package. (It would obviously also need to be part of the runtime deployment to be of any use.)
Of course, there's already the excellent PL/JSON project which implements a JSON parser as an object type (and comes with a number of JSON-related utilities), but it would be great to have JSON parsing capabilities built-in to any Apex application (ie any database with Apex installed).
Also, it would make sense to move the JSON-generating procedures (json_from_items, json_from_sql, etc.) from APEX_UTIL to APEX_JSON. Those procedures, by the way, are still undocumented as of Apex 4.2.2.
So, dear Apex team, please consider adding the APEX_JSON package as a supported, documented, runtime-deployed package in the next Apex release!
And dear blog reader, if you think this is a good idea, go to the Apex Feature Request application and vote for my request with identifier "AFZU".
Saturday, May 18, 2013
Friday, May 17, 2013
Using Apex with IIS Express
The Thoth Gateway is a gateway written in C# and ASP.NET that allows you to run PL/SQL web applications (including Oracle Application Express) on Microsoft's Internet Information Server (IIS). You can read more about the gateway here. It is an alternative to Apache/mod_plsql and the Java-based Apex Listener.
For production deployment on a server, you should use a regular IIS installation. However, if you are a developer working on a local Apex installation on your laptop, then the full IIS package is more than you need. Microsoft also offers a lightweight alternative in IIS Express:
Using the Thoth Gateway with IIS Express is in many ways similar to running the Apex Listener in Standalone mode. Here's how to use the Thoth Gateway with IIS Express on your development machine.
Run the installer.
This will put the IIS Express software under C:\Program Files\IIS Express and the configuration files under C:\Documents and Settings\Administrator\My Documents\IISExpress (assuming you are running Windows XP and your username is Administrator).
Copy and paste the following:
<application path="/" applicationPool="Clr2IntegratedAppPool">
<virtualDirectory path="/i" physicalPath="C:\temp\apex_4.1.1_en\apex\images" />
<virtualDirectory path="/" physicalPath="C:\temp\apex_4.1.1_en\apex\images" />
</application>
<application path="/web" applicationPool="Clr2IntegratedAppPool">
<virtualDirectory path="/" physicalPath="c:\temp\thoth_iisexpress" />
</application>
<bindings>
<binding protocol="http" bindingInformation=":8090:localhost" />
</bindings>
</site>
For better performance with static files, you should also add a httpExpires value to the clientCache setting:
<!-- see http://madskristensen.net/post/Add-expires-header-for-images.aspx -->
<clientCache httpExpires="Sun, 29 Mar 2030 00:00:00 GMT" cacheControlMode="UseExpires" />
Download the Thoth Gateway and unzip the file.
Instead of following the normal installation instructions (which assume a full IIS installation), simply create a folder (I've used c:\temp\thoth_iisexpress in the example above) and copy the web.config and the bin folder from the unzipped file to this location.
Create a batch file (I've called it start_iisexpress.bat) and put the following command in it:
"c:\program files\iis express\iisexpress.exe" /siteid:2 /systray:true
For production deployment on a server, you should use a regular IIS installation. However, if you are a developer working on a local Apex installation on your laptop, then the full IIS package is more than you need. Microsoft also offers a lightweight alternative in IIS Express:
IS Express (...) will run on Windows XP and higher systems, does not require an administrator account, and does not require any code changes to use.You might want to take a look at this nice video with lots of information about IIS Express.
Using the Thoth Gateway with IIS Express is in many ways similar to running the Apex Listener in Standalone mode. Here's how to use the Thoth Gateway with IIS Express on your development machine.
Download and Install IIS Express
Download the IIS Express installer from Microsoft's website.Run the installer.
This will put the IIS Express software under C:\Program Files\IIS Express and the configuration files under C:\Documents and Settings\Administrator\My Documents\IISExpress (assuming you are running Windows XP and your username is Administrator).
Configure IIS Express
Open \My Documents\IISExpress\config\applicationhost.config and add a new site under the "sites" node:Copy and paste the following:
<application path="/" applicationPool="Clr2IntegratedAppPool">
<virtualDirectory path="/i" physicalPath="C:\temp\apex_4.1.1_en\apex\images" />
<virtualDirectory path="/" physicalPath="C:\temp\apex_4.1.1_en\apex\images" />
</application>
<application path="/web" applicationPool="Clr2IntegratedAppPool">
<virtualDirectory path="/" physicalPath="c:\temp\thoth_iisexpress" />
</application>
<bindings>
<binding protocol="http" bindingInformation=":8090:localhost" />
</bindings>
</site>
Some points to note:
- This configuration sets up the virtual directories to serve the Apex images (in the "i" folder) and to run the actual gateway application (the "web" folder, which in mod_plsql is usually called "pls", but you can call it whatever you want). Both the images and the ASP.NET application are placed under c:\temp (but you can place them wherever you want).
- The port number is set to 8090.
- The traceFailedRequestsLogging has been disabled to improve speed.
For better performance with static files, you should also add a httpExpires value to the clientCache setting:
<!-- see http://madskristensen.net/post/Add-expires-header-for-images.aspx -->
<clientCache httpExpires="Sun, 29 Mar 2030 00:00:00 GMT" cacheControlMode="UseExpires" />
Download and Install the Thoth Gateway
Download the Thoth Gateway and unzip the file.
Instead of following the normal installation instructions (which assume a full IIS installation), simply create a folder (I've used c:\temp\thoth_iisexpress in the example above) and copy the web.config and the bin folder from the unzipped file to this location.
Create a shortcut to start IIS Express
Now we just need a convenient way to start IIS Express whenever we want to do Apex work.Create a batch file (I've called it start_iisexpress.bat) and put the following command in it:
"c:\program files\iis express\iisexpress.exe" /siteid:2 /systray:true
Create a shortcut to this batch file on your Windows desktop, and double-click the shortcut to start IIS Express.
As long as IIS Express is running, a console window will show each request:
That's it, you now have a lightweight, local web server for working with Oracle Application Express on a Windows machine.
Troubleshooting
If you need to troubleshoot the installation, you should enable the traceFailedRequestsLogging (see above). You'll find the trace files under \My Documents\IISExpress\TraceLogFiles. When you are done, disable this setting again for better performance.
Also note that all configuration changes to IIS Express require you to restart IIS Express before they take effect.
Sunday, February 17, 2013
Version 1.7 of Alexandria Utility Library for PL/SQL
The latest version of Alexandria, the utility library for PL/SQL, is now available for download. The previous version has been downloaded more than 2,000 times in the last 8 months.
The docs/readme.txt file contains a fairly detailed list of additions, enhancements and bug fixes. Below are some of the more noteworthy changes:
You can now choose between installing the full library (close to 50 packages as of this version), by running the main /setup/install.sql script, or you can install just the core set of packages via setup/install_core.sql and then choose additional sets of packages on top of that (for example, /setup/install_microsoft.sql or /setup/install_inet.sql and so on).
Jeffrey Kemp has submitted several functional enhancements to the S3 package which are incorporated into this version. Jeffrey has also done talks on the Alexandria Library in general and the S3 package in particular; the slides from these presentations are available here. Thanks for spreading the word, Jeff!
A couple of new kids on the block:
The docs/readme.txt file contains a fairly detailed list of additions, enhancements and bug fixes. Below are some of the more noteworthy changes:
Improved installation script
You can now choose between installing the full library (close to 50 packages as of this version), by running the main /setup/install.sql script, or you can install just the core set of packages via setup/install_core.sql and then choose additional sets of packages on top of that (for example, /setup/install_microsoft.sql or /setup/install_inet.sql and so on).
Amazon S3 package enhancements
Jeffrey Kemp has submitted several functional enhancements to the S3 package which are incorporated into this version. Jeffrey has also done talks on the Alexandria Library in general and the S3 package in particular; the slides from these presentations are available here. Thanks for spreading the word, Jeff!
New packages: Icalendar, URI templates, and image utilities
A couple of new kids on the block:
- ICALENDAR_UTIL_PKG: Lets you create Icalendar files from the database.
- IMAGE_UTIL_PKG: Get information (such as image width and height) from JPG/GIF/PNG image files. This package is based on code from Anton Scheffer.
- URI_TEMPLATE_UTIL_PKG: Use URI Templates to deal with "nice URLs" in PL/SQL and Apex web applications.
Labels:
Alexandria,
Amazon S3,
pl/sql,
Resource Template
Wednesday, January 16, 2013
PL/SQL and Microsoft technology
Many companies tend to have mixed environments, with Oracle databases (and PL/SQL) running business-critical back-office systems, as well as Microsoft products for front-office applications such as email, word processing and collaboration/file sharing (ie Outlook, Office and Sharepoint), and of course the desktop operating system itself (Windows, with Internet Explorer, Internet Information Server, all integrated via Active Directory).
Working in these mixed environments means that you, as an Oracle PL/SQL developer, frequently need to work/integrate with various Microsoft technologies from PL/SQL.
Over the last couple of years, I've written a number of blog posts on this topic. This post is just a convenient collection of links to these previous posts:
PL/SQL and NTLM
Oracle Application Express (Apex) and IIS
Microsoft Office and Office Open XML (OOXML)
Microsoft Exchange (Email, Calendar and Contacts)
Working in these mixed environments means that you, as an Oracle PL/SQL developer, frequently need to work/integrate with various Microsoft technologies from PL/SQL.
Over the last couple of years, I've written a number of blog posts on this topic. This post is just a convenient collection of links to these previous posts:
PL/SQL and NTLM
- An implementation of the NTLM protocol for PL/SQL, which you can use to call web services on servers with Integrated Windows Authentication
- NTLM Single Sign-On in Oracle Application Express (Apex), how to set up your Apex applications so that users will automatically be logged in without having to enter their usernames and passwords, useful for intranet sites
Oracle Data Provider for .NET (ODP.NET)
- Minimal, non-intrusive install of ODP.NET, or how to make ODP.NET "just work" without interfering with other software on the machine, with the smallest possible footprint
Oracle Application Express (Apex) and IIS
- Using the Thoth Gateway to run Apex on Microsoft Internet Information Server (IIS), a freely available, easy-to-install, high-performance port of mod_plsql that runs on Microsoft's web server, including IIS 6 and IIS 7
- Using Oracle Proxy Authentication with the Thoth Gateway, or how to pass your Windows credentials from the the web server on to the database server
Microsoft Office and Office Open XML (OOXML)
- PL/SQL package for working with the OOXML file format, or how to read and write OOXML documents (Word, Excel, Powerpoint) from PL/SQL
Microsoft Exchange (Email, Calendar and Contacts)
- PL/SQL package for working with Microsoft Exchange, or how to read and write email, attachments, calendar events and contact information from PL/SQL
Microsoft Sharepoint
- I have not yet had the need (or the desire...) to work with Sharepoint from PL/SQL, but it would be quite easy to create a PL/SQL library package to interface with the Sharepoint web services, similar to the Exchange package listed above. Perhaps I will revisit this topic sometime in the future... ?
Labels:
Exchange,
IIS,
NTLM,
OOXML,
pl/sql,
Proxy Authentication,
Thoth Gateway
Tuesday, November 6, 2012
Apex Plugin: Execute PL/SQL code and return content to page
Here is a Dynamic Action plugin for Oracle Application Express (Apex) that lets you execute PL/SQL code in the database, and then send output generated on the server (via calls to the "HTP.P" procedure) back to any element on the web page.
A more specific use case would be to respond to the clicking of a button by generating a fragment of HTML using PL/SQL, and then refresh a DIV on the page with that dynamically generated content. See the screenshot below for an example; notice that part of the first region has been updated with content generated on the server (using input from the user, in this case the name from a regular text input item). Every time the user clicks on the button, the line above the button is updated using an Ajax call to the server.
The second region illustrates another use case: Automatically refreshing part of the page at a fixed interval.
This is how the page has been set up:
The first region has a standard text input item, and a button. The button has a Dynamic Action that is triggered when the button is clicked. The click executes the PL/SQL code which generates the greeting.
The dynamic action is set up as follows:
We can see that the content generated by the PL/SQL code is returned into a page element called "my_div". This div is simply included as the static content of the HTML region:
The second dynamic action has been set up to trigger on "Page Load". The plugin is set up to call the PL/SQL code every 5 seconds (nice way to hammer your database with requests...) and return the content into another div, called "my_other_div". The mode has been set to "Append after", which means the content will be added to the end of the existing content, instead of replacing it.
Here is the setup for the second region:
Conclusion: This plugin combines the flexibility of Dynamic Actions with the power of PL/SQL for dynamic content generation.You can download the plugin here. Update 22.05.2016: This plugin is now in my plugin repository at GitHub.
A more specific use case would be to respond to the clicking of a button by generating a fragment of HTML using PL/SQL, and then refresh a DIV on the page with that dynamically generated content. See the screenshot below for an example; notice that part of the first region has been updated with content generated on the server (using input from the user, in this case the name from a regular text input item). Every time the user clicks on the button, the line above the button is updated using an Ajax call to the server.
The second region illustrates another use case: Automatically refreshing part of the page at a fixed interval.
This is how the page has been set up:
The first region has a standard text input item, and a button. The button has a Dynamic Action that is triggered when the button is clicked. The click executes the PL/SQL code which generates the greeting.
The dynamic action is set up as follows:
We can see that the content generated by the PL/SQL code is returned into a page element called "my_div". This div is simply included as the static content of the HTML region:
The second dynamic action has been set up to trigger on "Page Load". The plugin is set up to call the PL/SQL code every 5 seconds (nice way to hammer your database with requests...) and return the content into another div, called "my_other_div". The mode has been set to "Append after", which means the content will be added to the end of the existing content, instead of replacing it.
Here is the setup for the second region:
Conclusion: This plugin combines the flexibility of Dynamic Actions with the power of PL/SQL for dynamic content generation.
Labels:
Apex,
Apex plugins,
database-centric architecture
Monday, August 13, 2012
Database (schema) backup to the cloud with PL/SQL
In my last blog post I described using the DBMS_DATAPUMP and DBMS_JOB packages to automate database schema backups from the database itself, rather than using batch files to do so. I also mentioned that "since the files are written to the database server itself, you need some kind of process to move the files to another server or storage location, so the backups don't disappear along with the database if the database server itself is somehow lost or corrupted. [One] option is to upload the backup file from the database to some online ("cloud") storage service, such as Amazon S3."
Since I have previously written a PL/SQL API for Amazon S3, uploading your backup files to S3 is not very difficult. The high-level steps are:
Since I have previously written a PL/SQL API for Amazon S3, uploading your backup files to S3 is not very difficult. The high-level steps are:
- Create an export (dump file) of your database schema using DBMS_DATAPUMP (or the DATAPUMP_UTIL_PKG wrapper described in my last blog post).
- Save the export file to disk on the server.
- Upload the export file to S3.
- (Optionally) Delete the export file from the server.
- (Optionally) Send an email notification indicating if the upload was successful or not.
In the "demos" folder of the Alexandria Utility Library for PL/SQL you can find a package called DATAPUMP_CLOUD_PKG which does the work described above in a procedure called BACKUP_SCHEMA_TO_S3.
Here is how you can set up a call to DBMS_JOB to schedule regular backups of your database schema to Amazon S3 (note the double quoting of the string values):
declare
l_job number;
l_what varchar2(4000);
begin
l_what := 'datapump_cloud_pkg.backup_schema_to_s3(''your_amazon_id'', ''your_amazon_password'', ''your_amazon_bucket'', ''your_amazon_folder'', ''YOUR_ORACLE_BACKUP_DIR'', ''your_backup_file_name_'' || to_char(sysdate, ''yyyymmddhh24miss''), p_email_failure => ''your_email_address@domain'', p_email_success => ''your_email_address@domain'', p_version => ''10.2'', p_gmt_offset => 0);';
-- run job at the end of each day
dbms_job.submit (job => l_job, what => l_what, next_date => sysdate, interval => 'trunc(sysdate + 1) + 23/24');
commit;
end;
/
Security Tip: You may not like to expose your Amazon password in the job definition. What you can do to reduce the risk is to use the AWS Identity and Access Management (IAM) tools to define a separate backup user which only has upload ("put") permissions on your S3 folder, and no privileges to list file contents, delete or download files. This will limit the damage that someone with a stolen password can do.
Monday, July 23, 2012
Simple database (schema) backup using PL/SQL
Usually, database backups are the responsibility of DBAs, and not something that developers really care or think too much about. However, if you are a lone developer, or part of a small team that doesn't have the luxury of having a dedicated DBA, then you might have to deal with backups yourself.
There are several ways to make Oracle backups; what I will be concentrating on here is the "Data Pump Export" method. You may already be familiar with the command-line "expdp" command which allows you to create a dump (.dmp) file containing your database objects (schemas, tables, procedures, etc.).
To perform a regular backup, one could create a simple batch file to run "expdp" and schedule it using the Windows Task Scheduler or a Unix Cron job. While this certainly works, it means that you have one more "moving part" to think about (the OS scheduler), and you may have to hardcode database passwords in the batch file unless you use OS authentication.
However, the "expdp" command-line utility is actually just a "front-end" to a PL/SQL package called DBMS_DATAPUMP, as can be seen in this diagram:
This means that we can call the DBMS_DATAPUMP package directly from our own PL/SQL code to create dump files, and use DBMS_JOB to schedule this backup at regular intervals. We avoid the dependence on batch files, and we don't have to expose any passwords.
The API for DBMS_DATAPUMP is flexible and, as mentioned, supports everything that can be done from the command line (both exports and imports), but it can sometimes be a bit tricky to get the (filtering) syntax correct.
I've written a wrapper package called DATAPUMP_UTIL_PKG, which you can download as part of the Alexandria PL/SQL Utility Library. This package makes it really easy to perform the most common use case: Exporting a single schema to a file on disk. This can be accomplished with the following code:
-- export current schema to file, use default file name, and make the export compatible with XE 10g
-- include a custom message
begin
debug_pkg.debug_on;
datapump_util_pkg.export_schema_to_file ('DEVTEST_TEMP_DIR', p_version => '10.2', p_log_message => 'it is possible to include custom messages in the log');
end;
/
There are several ways to make Oracle backups; what I will be concentrating on here is the "Data Pump Export" method. You may already be familiar with the command-line "expdp" command which allows you to create a dump (.dmp) file containing your database objects (schemas, tables, procedures, etc.).
To perform a regular backup, one could create a simple batch file to run "expdp" and schedule it using the Windows Task Scheduler or a Unix Cron job. While this certainly works, it means that you have one more "moving part" to think about (the OS scheduler), and you may have to hardcode database passwords in the batch file unless you use OS authentication.
However, the "expdp" command-line utility is actually just a "front-end" to a PL/SQL package called DBMS_DATAPUMP, as can be seen in this diagram:
This means that we can call the DBMS_DATAPUMP package directly from our own PL/SQL code to create dump files, and use DBMS_JOB to schedule this backup at regular intervals. We avoid the dependence on batch files, and we don't have to expose any passwords.
The API for DBMS_DATAPUMP is flexible and, as mentioned, supports everything that can be done from the command line (both exports and imports), but it can sometimes be a bit tricky to get the (filtering) syntax correct.
I've written a wrapper package called DATAPUMP_UTIL_PKG, which you can download as part of the Alexandria PL/SQL Utility Library. This package makes it really easy to perform the most common use case: Exporting a single schema to a file on disk. This can be accomplished with the following code:
-- export current schema to file, use default file name, and make the export compatible with XE 10g
-- include a custom message
begin
debug_pkg.debug_on;
datapump_util_pkg.export_schema_to_file ('DEVTEST_TEMP_DIR', p_version => '10.2', p_log_message => 'it is possible to include custom messages in the log');
end;
/
To schedule this as a regular job, use the following code:
declare
l_job number;
l_what varchar2(4000);
begin
l_what := 'datapump_util_pkg.export_schema_to_file(''DEVTEST_TEMP_DIR'', p_version => ''10.2'');';
-- run job at the end of each day
dbms_job.submit (job => l_job, what => l_what, next_date => sysdate, interval => 'trunc(sysdate + 1) + 23/24');
commit;
end;
/
For this to work, you also need to set up a directory on the database server:
-- run as SYS
create directory devtest_temp_dir as 'c:\some_folder\backup\';
grant read, write on directory devtest_temp_dir to your_database_schema;
Obviously, since the files are written to the database server itself, you need some kind of process to move the files to another server or storage location, so the backups don't disappear along with the database if the database server itself is somehow lost or corrupted. Unlike the "hot" database files, these dump files can be safely copied by normal backup software.
Another option is to upload the backup file from the database to some online ("cloud") storage service, such as Amazon S3. I will talk about that in my next blog post...
PS. Don't forget to regularly test a restore (import) of your backup files. The only thing worse than not having a backup is having a backup that can't be restored... !
Labels:
Alexandria,
database-centric architecture,
pl/sql
Subscribe to:
Posts (Atom)












