Showing posts with label XE. Show all posts
Showing posts with label XE. Show all posts

Tuesday, October 14, 2025

Joel Kallman Day 2025: Making sure Oracle XE does not run out of space #JoelKallmanDay

 

 I'm writing a short blog post today as part of the Joel Kallman Day 2025.


Oracle Database Express Edition (XE) is the free edition of the Oracle Database. And some time ago, Oracle started calling it the "Free" database instead of the "XE" database, but it's basically the same thing.

It's free to develop with, and free to deploy to production. It has limitations, and there is no support except through the community, but otherwise it's a great database for many use cases.

One of the limitations is disk space. Your own user data (usually in the USERS tablespace) plus some system data (in the SYSAUX tablespace) cannot exceed 12 GB.


 

A potential problem is that since Oracle XE comes with a lot of the same bells and whistles as the Enterprise Edition (EE), there are many features that generate a lot of "unnecessary" data in the SYSAUX tablespace, at least in the context of a typical XE environment.

Connor McDonald has several blog posts, including this one, which go into this in great detail, so I won't repeat everything here.

The point of this blog post is just to let you know that in order to avoid unrestricted SYSAUX growth in your Oracle XE/Free Database, you need to change several default settings and turn features off. In addition to what Connor has listed in his blog post (see above), I have also found another component (the "Auto STS Capture Task") that needs to be disabled to avoid filling up the SYSAUX tablespace.

 

Here, then, is the script I use right after installing Oracle XE:

 

-- NOTE: this should be done in the pluggable database (XEPDB1 or FREEPDB1), not in the container database (CDB$ROOT)


alter session set container = xepdb1;


-- create a separate tablespace for audit info

create tablespace audit_tbs1 datafile '/opt/oracle/oradata/XE/XEPDB1/audit01.dbf' size 20m autoextend on next 2m;
 
begin
  dbms_audit_mgmt.set_audit_trail_location (audit_trail_type => dbms_audit_mgmt.audit_trail_aud_std, audit_trail_location_value=>'AUDIT_TBS1');
  dbms_audit_mgmt.set_audit_trail_location (audit_trail_type => dbms_audit_mgmt.audit_trail_fga_std, audit_trail_location_value=>'AUDIT_TBS1');
  dbms_audit_mgmt.set_audit_trail_location (audit_trail_type => dbms_audit_mgmt.audit_trail_db_std, audit_trail_location_value=>'AUDIT_TBS1');
  dbms_audit_mgmt.set_audit_trail_location (audit_trail_type => dbms_audit_mgmt.audit_trail_unified, audit_trail_location_value=>'AUDIT_TBS1');
end;
/
 
begin
  -- lower retention periods for various components
  dbms_audit_mgmt.set_last_archive_timestamp (audit_trail_type=>dbms_audit_mgmt.audit_trail_unified, last_archive_time=>sysdate-7);
  dbms_workload_repository.modify_baseline_window_size (window_size =>7);
  dbms_workload_repository.modify_snapshot_settings (retention=>7*1440);
  dbms_stats.alter_stats_history_retention (7);
  dbms_scheduler.set_scheduler_attribute ('log_history',7);
end;
/
 
-- disable Stats Advisor
begin
  dbms_stats.set_global_prefs('AUTO_STATS_ADVISOR_TASK','FALSE');
end;
/
 
-- disable capture of SQL workloads (only used for Automatic Indexing which is turned off by default)
 
begin
  dbms_auto_task_admin.disable (client_name => 'Auto STS Capture Task', operation => NULL, window_name => NULL);
end;
/

 

And if you use APEX (and who doesn't?!?!), you should also install APEX into its own tablespace:

 

-- NOTE: this should be done in the pluggable database (XEPDB1 or FREEPDB1), not in the container database (CDB$ROOT)


alter session set container = xepdb1;

 

-- create a separate tablespace for APEX (version 22.2 in this case)

create tablespace apex22_tbs1 datafile '/opt/oracle/oradata/XE/XEPDB1/apex22.dbf' size 20m autoextend on next 20m;

 

-- when you install APEX, specify the new tablespace

@apxins.sql APEX22_TBS1 APEX22_TBS1 TEMP /i/

 

 

Hopefully this will be useful to someone. Happy Joel Kallman Day! :-)

 

Wednesday, October 18, 2017

Oracle XE 12c becomes Oracle XE 18c

Oracle Database, Express Edition (XE) is a free version of the Oracle Database, currently available as version 11g (11.2) for Windows and Linux.
A planned new version of Oracle XE based on 12c (12.2) was first mentioned back in 2013. Now we are in 2017 and Oracle has changed its version numbering, aiming for yearly releases of the database, which means that after 12c comes 18c (in 2018) and 19c (in 2019) and so on.

According to information coming out at this year's Oracle OpenWorld, the next version of Oracle Database Express Edition (XE) will therefore be Oracle XE 18c.

Below are the details known so far about this upcoming version. Disclaimer: I've collected this information from tweets and blogs, so nothing is official yet. Time will tell as to what is true. UPDATE: Gerald Venzl from Oracle has confirmed the below to be "all true" :-)

  • The next version of Oracle Express Edition (XE) will be 18c. (Source: Chris Saxon, Twitter)
  • Oracle XE 18c is expected in Q1 of 2018. (Source: AMIS blog). UPDATE: Oracle XE 18c "is currently planned between March and August 2018 and might change". (Source: Gerald Venzl, Twitter)
  • There will be yearly releases of Oracle Express Edition (XE), ie Oracle XE 19c in 2019, etc. (Source: Franck Pachot, Twitter).
  • There will be simultaneous releases of XE for Linux and Windows. (Source: Gerald Venzl, Twitter)
  • Limits for XE 18c will be 2 GB of memory, 12GB of storage (with basic/advanced compression bringing real capacity up to around 40GB), 2 CPUs and 4 pluggable databases. (Source: AMIS blog and Lucas Jellema, Twitter)
  • Express Edition (XE) will actually include "nearly all" of the features from Enterprise Edition (EE)! (Source: Franck Pachot, Twitter and Chris Saxon, Twitter).
  • Express Edition (XE) will still be free for both development and production. (Source: Chris Saxon, Twitter).
  • There will be no support (except through community/forums) for XE, and no bug fixes/patches. Still, with a yearly release cycle that means bugs will be fixed by upgrading to the latest release. (Source: Franck Pachot and Bob Bryla, Twitter)


If most, or even some, of the above is true, this is really great news! I understand we should thank Gerald Venzl at Oracle for this, as he is the guy working on bringing us all this goodness! Thanks in advance, Gerald! :-)





Monday, October 10, 2016

OTN Appreciation Day: Oracle Express Edition (XE)

Tim Hall of oracle-base.com fame suggested we should all do an "OTN Appreciation" day in honor of the Oracle Technology Network (OTN), by blogging about our favorite Oracle product or feature.


My personal favorite is Oracle Express Edition (XE), the free version of the world's best relational database.



 Here's what's great about it:


Oracle Express Edition is currently available in version 11gR2 (11.2). Based on hints dropped by various folks at Oracle, there will be a 12c version of Oracle XE, but it is probably a year or two away yet.

The XE documentation is a great starting point for those new to Oracle, with several "2-day guides" to database administration and development tasks.

Enjoy Oracle XE and #ThanksOTN for this great product! :-)

Tuesday, April 26, 2016

How to set up IIS as SSL proxy for utl_http in Oracle XE

If you want to call a web service over HTTPS from the utl_http or apex_web_service packages in PL/SQL, you need to set up an Oracle Wallet that contains the SSL certificates of the server you are connecting to from the database.

Setting up an Oracle Wallet is quite straightforward, but it can be a bit of a hassle to configure a large number of certificates. Also, if you are using Oracle Express Edition (XE) which gets very infrequent updates, you are stuck with whatever SSL protocol support was in the database at the time of release.

One solution is to set up an outgoing proxy in a local web server. The database will communicate with the web server via plain HTTP, without any need for SSL certificates, and the web server translates the requests to HTTPS. This is fine from a security point of view as long as the database and the web server are on the same machine or on the same internal network. Traffic from the web server to the outside world is encrypted using SSL, while the internal traffic between the web server and the database remains unencrypted.

Richard Martens wrote a blog post about this and showed how to set up the proxy on Apache. In this blog post I will show you the equivalent setup on Microsoft Internet Information Server (IIS).

Step 1: Installation

First you need to download and install the following IIS extensions from Microsoft:
Install these extensions as per Microsoft's documentation.

Step 2: Set up proxy website

Create a new website in IIS. The name of the website is not important, but in this example I will call it "MyProxy". You need to link the website to a physical location on disk, I use C:\temp\myproxy but again this does not really matter.



Then edit the website bindings to bind the site to a specific IP address and port. In this example, I choose 8888 as the port number. I also bind the site to the localhost IP address 127.0.0.1. This ensures that the site can only accept connections coming from the local machine, and therefore assumes that my Oracle database is located on the same machine as the web server.

Step 3: Configure URL rewrite from HTTP to HTTPS

Follow the instructions in this blog post to enable Application Request Routing at the server level (short version: Click "Application Request Routing Cache" at the server level, choose "Server Proxy Settings" and then check the "Enable Proxy" checkbox).

Then go to the MyProxy website that you created in step 2, and open the "URL Rewrite" feature. Create a new rule called something like "Forward all requests as HTTPS".


Set up a rewrite rule as follows: Pattern myproxy/(.*?)/(.*) rewrites to https://{R:1}/{R:2}


This rule means that, for example, a request to http://127.0.0.1:8888/myproxy/example.com/foo/bar would be rewritten to https://example.com/foo/bar


Step 4: Test the proxy setup

You can now try out the proxy rewrite rule from a Powershell command prompt on the server. In the screenshot below you can see two regular requests (the direct HTTP request doesn't work as the remote server does not accept non-HTTPS requests), as well as a request to the local proxy which gets a successful response back from the remote server.



With that in place, we can now turn to the database.

Step 5: Adjust database network ACL

To be able to access the local proxy, we need to open this up using the database network ACL. A typical script to do this would look like this:

-- to be run by user SYS

alter session set nls_language = AMERICAN;
alter session set nls_territory = AMERICA;

BEGIN
  DBMS_NETWORK_ACL_ADMIN.create_acl (
    acl          => 'apex.xml',
    description  => 'Access Control List for APEX',
    principal    => 'APEX_050000',
    is_grant     => TRUE,
    privilege    => 'connect',
    start_date   => SYSTIMESTAMP,
    end_date     => NULL);
  COMMIT;
END;
/

-- for https proxy
-- see http://blog.rhjmartens.nl/2015/07/making-https-webservice-requests-from.html
-- see http://ora-00001.blogspot.com/2016/04/how-to-set-up-iis-as-ssl-proxy-for-utl-http-in-oracle-xe.html

BEGIN
  DBMS_NETWORK_ACL_ADMIN.assign_acl (
    acl         => 'apex.xml',
    host        => '127.0.0.1',
    lower_port  => 8888,
    upper_port  => 8888);
  COMMIT;
END;
/


Since we can now use this proxy for any HTTPS traffic, we only need this single entry in the network ACL. (If you want to narrow down the list of allowed remote sites, you could adjust the rewrite rule in step 3 so that it only works for specific sites instead of all sites.)

Step 6: Enjoy

You can now use the utl_http and apex_web_service packages from PL/SQL to call any HTTPS site. Just remember that you need to alter the original URL so it hits the proxy instead.



Thursday, July 23, 2015

ORA-22926 when using getClobVal to convert XMLType to CLOB

I ran into a problem the other day when moving some code from one database to another (both XE 11g). The code in question needs to convert an XMLType to a CLOB to do some (hacky) string manipulation on it, and then turn it back to an XMLType.

Here's the original code:

  l_xml := apex_web_service.make_request(...);
 
  -- little hack to remove bad empty namespace from result
  l_clob := l_xml.getClobVal();
  l_clob := replace (l_clob, '<mytag xmlns="">', '<mytag>');
  -- and then strip out bogus namespace to make parsing easier...
  l_clob := replace (l_clob, ' xmlns="http://tempuri.org/"', '');
 
  l_xml := xmltype (l_clob);


The above had been working before, but now failed with "ORA-22926: specified trim length is greater than current LOB value's length".

Googling turned up a forum post which pointed out that getClobVal is deprecated, and the suggestion is to use XMLSerialize instead.

This can only be used in a SQL statement, not as a PL/SQL expression, so I had to rewrite as follows:

select xmlserialize(document l_xml as clob)
into l_clob
from dual;


And the error went away... (It seems some people have had problems even with the xmlserialize function, but that's quite an old post, so that particular bug could have been fixed a long time ago.)

Thursday, July 16, 2015

Installing Oracle XE, ORDS and Apex on CentOS - Part Four: Stress testing

This is part four in a series of blog posts about how to install Oracle 11g Express Edition (XE) with Oracle Application Express (Apex) on a CentOS Linux server, with Apex served by Oracle REST Data Services (ORDS) running on top of Tomcat and Apache.

After setting up a small CentOS server with Oracle Express Edition (XE), I wanted to stress test it to see how much load such a barebones installation can handle. In other words, how many users can you serve using this minimal, license-free setup?

The test application

I set up a test application in Apex with a single page that has both some static and dynamic (PL/SQL) regions, as well as a query and some processes. I set the page authentication to "No authentication" as the various testing tools need to be able to access this page without logging in (in which case we'd have to make a more complex test script).


Just checking the page via a web browser we can see that it is delivered fairly quickly from the server, in less than 250ms, and that static resources are cached. (Not all automated load testers respect cache settings, though. More about that later.)



First tests - BlazeMeter.com


For testing, I started with BlazeMeter.com, as they have a free plan that can stress test a site with up to 50 concurrent users. This is probably more concurrent users than most business applications built for small and medium companies will have/need, and so the free test works well for our purposes. (This obviously depends on your definition of small/medium company, but if you have so many users/employees, then why are you using Oracle XE? :-)

Setting up a stress test using BlazeMeter is easy, although the user interface is a bit cluttered with advanced options that can all be ignored for simple tests. Because we only want to load test a single page, we need to create a so-called "URL Test", and give it a name and specify the URL. Select an appropriate location to run the test from, the number of users (50 is the max for the free account) and the duration (the default is 20 minutes, which may be a bit long for simple tests; you can set this to 5 or 10 minutes for a quick test).






Running the first test

Having set up the test page in Apex and the test itself in Blazemeter, I started the test and watched it... crash and burn! :-( This is what the test results looked like:



As you can see from the chart above, more than a third of the requests resulted in errors returned from the web server, and the average response time is almost 10 seconds! Not good! I checked the Tomcat logs (at /usr/share/tomcat7/latest/logs if you have followed the setup in this series of blog posts) and found lots of this message:

java.sql.SQLException: Exception occurred while getting connection: oracle.ucp.UniversalConnectionPoolException: All connections in the Universal Connection Pool are in use

Turns out the default ORDS connection pool size is too small. I found this article which gives some advice. I changed the following in the ORDS configuration (/u01/ords/config/ords/conf/apex.xml if you have used the config location suggested in my previous blog posts) and restarted Tomcat:



Re-running the first test


Re-running the test gave the following result:



Much better! No errors, and the response time is pretty good for this minimal server. Remember it only has 1 CPU and 1GB of memory, running Oracle XE, Tomcat and Apache. And yet it quite happily serves 50 concurrent users (average of 3,5 requests per second) without any problems.

Scaling up


One of the nice things about DigitalOcean (and other cloud server providers) is that it is easy to scale up (and down) the server as needed. I decided to re-run the same test using a server with 2 CPUs and 4 GB of memory (although Oracle XE is limited to 1 CPU and 1 GB of memory, it should free up some more resources for the web server and other OS processes).



The test results on this somewhat bigger server shows a flatter response time curve, compared to the 1GB server which had a couple of random peaks. The average response time is less than 700ms, compared to 950ms for the smaller server. We are still limited to 50 users (average of 3,5 requests per second) since this is the maximum that BlazeMeter's free plan allows.



More testing - LoadImpact.com


I did further tests with LoadImpact.com, because they have a free plan with up to 100 concurrent users (or VUs, "Virtual Users", in their terminology). This is the test result with 100 concurrent users, run against the minimal server which been scaled back down to just 1 CPU and 1 GB of memory.



As the chart indicates, the response time remains more or less flat regardless of the number of concurrent users.

Never mind that the chart indicates that each page view took several seconds to complete. The FAQ page of LoadImpact states that "simulated clients in a test will never cache anything (except for cookies). This means that in a test, every client that loads a page from your site will behave like a new visitor to the site and thus be quite 'heavy' on the server." Which means that actual page load times (when client caching is enabled) will be significantly better than the tests indicate, since static files don't have to be downloaded every time.

In other words, the flat response time curve means that the application scales very well, and could probably support many more users. Quite impressive, really, for a setup that costs just USD 10 per month! :-)

Even more tests - loader.io


Finally, I tested with loader.io, who offer a free plan with up to 10.000 (!) clients in 1-minute tests. Running a number of different tests (and also bumping up the jdbc.MaxLimit setting from 60 to 100), I found that the "breaking point" for the server when using my test page was around 12-15 requests per second, which still gave sub-second response times. If pushed any further, the response times would quickly go up to several seconds.



I also ran some tests against a very barebones Apex page that shows nothing but a static HTML region.



When stress testing against this page, the server could handle up to around 25 requests per second while still maintaining sub-second response times. So obviously scalability depends on what you put on your pages.



Conclusions

Real-world performance will depend on a lot of factors. The test page I used was fairly typical for a business application in that it has a report, some dynamic PL/SQL content, and also does an insert into a table (so it's not just read-only). In a typical application, there will be pages that are more complex than this, and pages that are simpler than this, so it should average out.

So let's assume that the server can handle 15 requests per second with acceptable response times. What does that translate to in real-world terms? Here are some quick calculations (I'm assuming most Apex business apps will be used during an 8-hour office hour period each day):

  • 450 users doing 1 000 page views each in an 8-hour period per day (450*1000/8/60/60 = 15 requests per second)
  • 1 500 users doing 250 page views each in an 8-hour period per day (1500*250/8/60/60 = 13 requests per second)
  • 10 000 users doing 45 page views each in an 8-hour period per day (10000*45/8/60/60 = 15 requests per second)
  • 25 000 users doing 50 page views each in a 24-hour period per day (25000*50/24/60/60 = 14 requests per second)

Now remember, the above is based on a single server with 1 CPU and 1GB of RAM, running Oracle Express Edition (XE), Apache and Tomcat, and costing USD 10 per month! If this is not good value for money, I don't know what is...



Monday, July 13, 2015

Installing Oracle XE, ORDS and Apex on CentOS - Part Three: Additional configuration

This is part three in a series of blog posts about how to install Oracle 11g Express Edition (XE) with Oracle Application Express (Apex) on a CentOS Linux server, with Apex served by Oracle REST Data Services (ORDS) running on top of Tomcat and Apache.

By now, you should have a working installation of CentoOS with Oracle XE, Apex, ORDS, TomCat and Apache. There are still some additional configuration and optimization we should do, which we'll take a look at in this third installment of the blog post series.

This part is a "grab bag" of various tips. None of these steps are really required for a working installation, but most of them are strongly recommended for better security and performance.

Disabling root login

So far we have been using the root user to login to the server and perform administrative tasks. For various reasons it is better to disable logins for the root user and instead create an alternative user that can elevate its privileges to root level only when necessary.

This is explained in detail in this article.

Cleaning up Oracle XE

Let's clean up and disable/remove stuff we don't need from Oracle XE:


Optimizing Oracle XE

Let's change some default session, process and memory settings in Oracle XE:


Setting up Oracle XE network ACLs


If you plan to do any network calls from the database, for example to invoke web services, you need to explictly open the database Network Access Control List (ACL) to enable outgoing traffic to certain hosts/IP addresses and ports.

This is explained in detail in this article. Here is a sample script that can be used as a starting point:


Closing XDB port (8080) in the firewall


Remember to close port 8080 in the firewall, as it is not be needed in the default software stack we have set up. Revisit the firewall.sh script we created earlier and make sure that the line that opens port 8080 has been commented out or deleted from the firewall script. Re-run the script if necessary to make sure the change takes effect.


Optimizing ORDS


The default connection pool settings in the ORDS configuration are too small. You'll have to experiment to see what the best settings are for your workload, but the following seem to work well. Edit the /u01/ords/config/ords/conf/apex.xml file and put in the below JDBC settings. Restart Tomcat for the changes to take effect.



Cleaning up Tomcat


Let's remove the sample apps from the default Tomcat install, as we won't be needing any of it:



Cleaning up Apache


Let's remove the default Apache home page and replace it with our own. Do this by creating an index.html page on your local computer and then copy it to the correct folder on the server:



Optimizing Apache


Well, not so much optimizing Apache itself, but optimizing delivery of website content by making sure that the contents is being compressed and cached, which will speed up performance.

Actually, we already did this as part of the Apache installation (in part two of this blog post series). If you look back at the apex.conf file we created, it included configuration for the modules mod_deflate (for compression) and mod_expires (for caching).

You can verify that compression is enabled by looking at the headers returned from the server via Chrome's network tab (look for the Content-Encoding header):



Also, if you enable a two-line display of each request, Chrome will show the uncompressed file size as well as the compressed file size, so you can see how much bandwidth was saved.



You can verify that caching is enabled (for static files such as images and Javascript) by looking at the Expires header, which means that the browser should keep a copy of this file and not download it again until after it has expired:



I usually put an expiration of 7 days on static files. This means that typical business applications that are used daily or several times per week will always have a copy of the static files already downloaded, which speeds up things considerably. And if you do change the files (as part of an application upgrade), then it will take a maximum of one week before all your users have the latest files. (Tip: Users can always force a new download of cached files by holding down the Shift key and reloading the page in the browser.)

Chrome shows files retrieved from the cache as "(from cache)" and no time (0 ms) is spent downloading the file.



Adding robots.txt


Add a robots.txt file to the root of your website to avoid search engines hitting your website and wasting its resources and bandwidth. This assumes that you mostly have private (internal business) Apex applications that require logins, and since the search engine will not be able to login, we might as well tell the search engines to not even bother trying.


Here is a sample robots.txt that advises search engines to avoid certain folders. Note that the robots.txt file in no way blocks access, so don't try to use this to "secure" content.

If, on the other hand, you have applications with public pages that you would like to have indexed by search engines, then by all means change or skip the robots.txt file altogether. (Or look into the use of sitemaps to direct the search engine towards the important pages of your website.)


Setting up SSL certificates


Any website that requires authentication should be running via HTTPS to encrypt the communication between the client and the server. This means you need to install an SSL certificate.

For development environments, you could use a self-signed certificate, which is free. The disadvantage is that browsers will complain (some more than others) about the self-signed certificate being unsafe because it is "untrusted", but as long as you know who installed the certificate (you!), it's fine and you can ignore the warnings.

Installing mod_ssl and configuring a self-signed certificate is explained in this article. Here is a short version:



Modify the default configuration to disable weak/broken ciphers and protocols, and also to set up the port forwarding of the SSL port (443) to Tomcat/ORDS:



For production environments you need an SSL certificate issued by a Certificate Authority (CA). There are many CAs and lots of companies offering SSL certificates. I won't go into the details of buying a certificate here, but pick a cheap one -- there is really not much technical difference between the cheap certificates and the very expensive ones.

After you have set up the SSL certificate on your website, you should test that it works correctly by running the SSL test at ssllabs.com. You should aim for nothing less than a "Grade A" result. If you are using a self-signed certificate, the test will complain about the certificate not being trusted, but you still get the SSL settings of your site verified.



Setting up backups


There are two parts to this: Backing up the server itself, and backing up the database. The latter is the most important to me (as the operating system can always be reinstalled quite quickly, as this series of blog posts show, but the database of course contains the real value -- your data!).

A cloud server provider such as DigitalOcean offers some built-in mechanisms for backups and snapshots, as explained here.

Also, you could use the rsync command to set up your own backup of the OS files.

For backing up the database (schemas), see this blog post I wrote a couple of years ago about using PL/SQL to do Data Pump exports and transfer the files to Amazon's Simple Storage Service (S3).


Monitoring server uptime


Having spent all this effort to set up your website, you want to make sure it is up and available to your users. You should set up one or more automated uptime monitors, using a free service such as Pingdom or UptimeRobot. These services will hit your server from various locations around the world at regular intervals, and send you an email or SMS if the site is unavailable. They will also keep statistics of the average response times and the uptime percentage, which is useful to analyse your server performance over time.




Scaling up (or down) the server


At some point you may want to scale up your server by adding more CPUs, disk space, memory and/or bandwidth. If you are using a cloud service such as DigitalOcean, this is very easy to do. Simply shut down the server, select a new server size from the control panel, and restart the server. This is explained here, and the process will be similar for other virtual server environments.

Note that if you scale down the server and thereby reduce the amount of available memory, you might get an ORA-00845: MEMORY_TARGET not supported on this system error from Oracle and you might need to adjust the operating system memory settings as described in this article. The short version is as follows:


Keeping the server updated

You should keep your server up-to-date with the latest OS patches. You can manually run yum list updates to see what's available, and yum update to download and install all relevant operating system updates that are available. (On DigitalOcean, you will get a warning that says "grubby fatal error: unable to find a suitable template" that has to do with the way Linux kernels are managed in DigitalOcean. You can ignore the warning. If you need to update the kernel, this is done via the server control panel, as described here.)

To set up automatic updates, install the package yum-cron and configure it (described in more detail here).



Next steps

 
That concludes this collection of additional configuration. The next (and final) blog post in this series will deal with performance testing the new server to verify that it can handle an appropriate number of users.


Monday, June 15, 2015

Installing Oracle XE, ORDS and Apex on CentOS - Part Two: Installation

This is part two in a series of blog posts about how to install Oracle 11g Express Edition (XE) with Oracle Application Express (Apex) on a CentOS Linux server, with Apex served by Oracle REST Data Services (ORDS) running on top of Tomcat and Apache.

Let's get right to it, starting with Oracle XE.

Installing Oracle XE

This is actually quite straightforward, assuming you have followed the prerequisite steps in part one.



As part of the installation, we created a user called oracle. It is useful to set up the default environment of this user to include the path to the sqlplus executable, so we can start sqlplus from anywhere.



Now, let's log in to Oracle as SYS and check that everything looks OK:



At this point you have an Oracle XE instance running, which also includes Apex 4.0 and the Embedded PL/SQL Gateway (EPG) running on port 8080. That is nice, but it's an old Apex version and the EPG web server is not really suited for heavy usage. We want the latest Apex version, and we want to use ORDS. Read on...

Installing Java


ORDS and Tomcat are both Java applications, so we need to install Java. Actually, we need the Java JDK (Java Development Kit), as opposed to just the Java JRE (Java Runtime Environment). There may already be something called the OpenJDK on the CentOS server, but we want the Oracle-supplied JDK, so let's remove OpenJDK and install the JDK that we downloaded from Oracle:

To have the Java binaries available from anywhere, we add the Java path to the bash profile of the root user:


Installing Tomcat


To install Tomcat, we will download the installation file directly to the server using the wget command, and then unzip it. Create a tomcat user to run the tomcat process.

To avoid conflicts with Oracle XE running the Embedded PL/SQL Gateway on port 8080, change Tomcat's default port number to 8090 by editing the server.xml file. (Note: Because we will put Apache in front of Tomcat, we won't ever access Tomcat directly on port 8090, and we will soon disable EPG, but let's just avoid possible conflicts anyway by assigning different ports.) It's also important to set the URIEncoding to UTF-8.

Next we need to create a script that can be used to start, stop and restart Tomcat as a service. Save the following as tomcat under /etc/init.d/


Then we need to set up the above script to run automatically if the server is rebooted.


Installing ORDS


The Oracle Rest Data Services (ORDS) installation consists of unzipping the installation file, running the configuration to specify database details, and then copying the ords.war file into the Tomcat webapps folder.


Installing Apache


The last step in completing our web stack is to install the Apache HTTP server and place it "in front of" Tomcat. This means that all requests to the server go to Apache first. Requests for static files (images, Javascript and CSS) is served directly by Apache. Requests for dynamic content (ie the actual HTML pages generated by Apex via ORDS) is served by Tomcat, using Apache as a proxy.

Installing Apache is very straightforward:

Then we need to add our custom configuration. By default, Apache is set up to read any .conf file placed in the /conf.d/ subfolder, so let's create an apex.conf file there. Note that these additional config files are read and processed in alphabetical order, so name your custom config accordingly if you use multiple config files.


Installing (upgrading to) latest Apex version


Finally, we need to upgrade the Apex installation that came bundled with Oracle XE to the latest and greatest Apex version (version 5.0 at the time of writing).

This is done by unzipping the Apex installation file, then running the Apex installation script via sqlplus. There are two different Apex installations to choose from: Either a full installation that includes the Application Builder (suitable for a development environment), and a more lightweight and secure "runtime-only" installation (suitable for test and production environments). Running the full installation on the standard 1GB server at DigitalOcean should take about 12-15 minutes.


We also need to make sure the apex_public_user schema is unlocked (and stays that way!).

We need to copy all the static Apex files (images, CSS, Javascript) to the Apache web folder.

When running on top of ORDS, Apex 5 uses the "RESTful Service" feature to serve any application-specific or workspace-specific static files, so we need to configure Apex with REST:

Now (finally!), if everything works, we should be able to access the new Apex installation by going to the following URL:

  http://servername/ords/apex

If everything works, you should see this familiar page:



Did it work? Great, now enjoy Apex 5! But wait, we are not fully done yet! In the next part of this series, I will describe various additional configuration that you should perform for a more secure and scalable server.

Stay tuned!

Tuesday, June 9, 2015

Installing Oracle XE, ORDS and Apex on CentOS - Part One: Preparation

This is part one in a series of blog posts about how to install Oracle 11g Express Edition (XE) with Oracle Application Express (Apex) on a CentOS Linux server, with Apex served by Oracle REST Data Services (ORDS) running on top of Tomcat and Apache.


Provisioning the server


First of all, we need a server to install the software on. There are many hosting providers that offer cheap Linux servers. I have been trying out DigitalOcean, where you can get a nice little server suitable for Oracle XE for as little as USD 10 per month. After you sign up, a new server complete with the operating system installed can be up and running in as little as 50 seconds (!). It is also very easy to scale up (or down) the server according to your needs.

A note on security


Linux security is a big topic in itself. In order to keep this blog post short and sweet, I will describe some security recommendations in a separate blog post. But in general, always use strong passwords (and/or certificates and/or two-factor authentication), never install more components than you need, and always apply the principles of minimal privileges and defense in-depth.

Creating the server


First you need to sign up to DigitalOcean (or another cloud provider of your choice). The sign-up process is quick and painless. I like that you can pay DigitalOcean using PayPal, so you don't need to give them your credit card details.




After signing up, create a new server (called a "droplet" in DigitalOcean's terminology).
When choosing a server size, keep in mind that Oracle XE cannot use more than 1 GB of memory. As we will see later, you can happily run Oracle XE together with Apache, Tomcat and ORDS, all on a 1GB server. (It is tempting to compare that to, say, the minimum requirements for Sharepoint... but let's stay on topic! :-)



You get to choose the physical location of the server. Select a data center that is geographically close to you and your users, as this will obviously reduce data transfer times across the network.

Select the 64-bit version of CentOS 6.5 (while CentOS 7.x is also available, this is not in widespread use yet, so I'm sticking with the more well-known version 6.x for now).


Logging in to the new server


When the server is created, you get the "root" password emailed to you, so you can login to the server. The root user on Linux is similar to the local Administrator account in Windows. As we will get back to later, it is actually recommended not to use the root user regularly, and only "become root" (via the "sudo" command or via other users defined as administrators in the "sudoers" file) when necessary. To keep things simple, we'll use the root user for now, but get back to security best practices in a later post.

Logging in to the new server is done via "ssh" (secure shell). If you have a Mac, just open a terminal and type ssh root@server_ip_address and when prompted enter your password to login.

If you have a Windows machine, there is no built-in ssh client, so you need to install some additional software. I recommend pimping up your Windows console window with cmder, a good-looking console emulator (make sure you download the full version of cmder that bundles the "msysgit" suite which includes a bunch of Unix commands, including ssh). Once you have cmder installed, use the same command as for the Mac above to login.

If you managed to login as root, run the command ls -la / and you should see something similar to this:



Congratulations, you now have your very own Linux server to play around with! :-)

Check out these links for more information:

Installing some basic utils


After logging in to the new server for the first time, let's install some basic utilities that we will need later. In CentOS, the "package manager" that you use to download and install software from a standard repository is called "yum". Run the following commands:


The basic utilities include "nano" (a text editor which I prefer over the default "vim" editor; the latter is probably very productive for power users, but a pain in the ass to use unless you remember a series of cryptic commands -- a bit like Linux in general I guess!), as well as "wget" (to get/download files from the Internet using the command line), "unzip" (self-explanatory) and "bc" (a basic calculator).

Setting the server time zone


It's useful to have the server date and time automatically synchronized based on a remote server, so let's set up Network Time Protocol (NTP).


Adding swap space


Oracle XE needs a certain amount of swap space (a file where the operating system can "swap" stuff from memory to disk when there is too little physical memory available), so let's set that up.


Setting up the fully qualified domain name (FQDN)


The "hosts" file contains a mapping between IP addresses and domain names. We need to make sure that the server's IP address is mapped to a "fully qualified domain name" (FQDN), otherwise the Oracle XE installation will fail, as described in detail here.

So, to make sure we have a fully qualified domain name in our hosts file, run nano /etc/hosts and put in a line with your IP address and your server name.


Setting up a firewall


We should only allow access to the server on the ports that we need. For a typical web server, this will be ports 80 and/or 443 (for HTTP and/or HTTPS), and port 22 (for SSH). Actually, you should change the default SSH port from 22 to some other random number, but we will get back to that later when we talk about hardening the server.

Tim Hall has written an excellent article about the Linux firewall, known as "iptables". Rather than repeat what Tim has already explained, I suggest you read his article, and then copy the following and save it as firewall.sh in the /root folder of your server. Then make the file executable by running chmod u+x /root/firewall.sh and then type ./root/firewall.sh to run the script.


Uploading Oracle software to the server


As the final part of preparing to install Oracle XE, ORDS and Apex on the server, we need to upload the different installation files to the server, so we can run them there.

Lets create a folder on the server where we can put the installation files:

mkdir -p /u01/download

The -p flag makes it possible to create two (or more) folders with a single command.

Why is the root folder called "u01"? Turns out this is a naming convention for Oracle software that goes way back. Linux folder names don't always make much sense, but I guess "u01" is as good as any.

And I guess there is an argument to be made for the sub-folder to be called "upload", as we will copy, or upload, stuff into it, but because it holds installation files that we would normally download directly to the server, I have called it "download".

The thing is, when downloading stuff from Oracle, you generally have to click an "Accept License Agreement" radio button and also login with an OTN account to actually get access to the file you wish to download. If it wasn't for this, the "wget" command could be used from the command line on the server to download the files directly to the server. As things stand, I find it best to download the desired software using a regular web browser on the client computer (ie my laptop) and then use the "scp" (secure copy) command to upload the files to the server.

So, go download the following software to your local computer:
  • Oracle Express Edition (XE) 11g for Linux x64
  • Java JDK 1.7 for Linux x64
  • Oracle Rest Data Services (ORDS) 2.0.10
  • Oracle Application Express (Apex) 5.0
I won't provide download links, as Oracle often changes the URLs on its web page and thus breaking the links, but a Google search should easily find these files on OTN. Download these to your local computer. Then copy the files to the server. If using a Mac, simply open a terminal window and use the scp command to copy the files:

If using Windows, either install a Unix emulation package (such as CygWin, cmder, msysgit, etc) that includes the scp command, or use the free WinSCP program if you prefer a graphical interface.

You should now have the installers for the various Oracle applications sitting in the /u01/download folder on the server.

Next Steps


Finally, we are now ready to install the Oracle software! This will be covered in part two of this series of blog posts.

Monday, June 8, 2015

Installing Oracle XE, ORDS and Apex on CentOS

I am writing a series of blog posts about how to install Oracle 11g Express Edition (XE) with Oracle Application Express (Apex) on a CentOS Linux server, with Apex served by Oracle REST Data Services (ORDS) running on top of Tomcat and Apache.

This is perhaps better explained with an illustration of the setup:

Best of all, this setup consists only of free (license-free) software, and as you will see in this series of blog posts, you will be able to run it all on a cloud server for as little as USD 10 per month, serving hundreds of concurrent users. Gotta love Apex! :-)

The articles will be divided up as follows:

These blog posts will assume that you are familiar with Oracle and Apex, but that (like me) you are a relative newcomer when it comes to Linux. Keep in mind that with Linux, there are typically many alternative ways to accomplish things, and I have chosen the approaches that seem most straightforward to me. Your mileage may vary.

Stay tuned for part one!

Wednesday, June 25, 2014

Oracle XE 11g Win64 on Microsoft Azure cloud


(Or: "How to build your own Oracle Cloud in 25 minutes"... ! )

After downloading the recently released Oracle Express Edition for 64-bit Windows, I decided to test it out on a Windows Server 2008 R2 running in the Azure cloud (recently renamed from "Windows Azure" to "Microsoft Azure", I guess because you can run Linux on it, as well).



Here are the high-level steps needed to get this up and running:

  1. Sign up for Azure and login to the management portal.
  2. Create a new Virtual Machine, select Windows Server 2008 R2. The provisioning of the VM takes around 5 minutes. (I chose to use an A1 instance type, which costs around USD 50 per month, and has 1 core and 1,75GB of memory. Note that XE is limited to use only 1GB of memory, leaving some extra memory for the OS itself and for IIS).
  3. When you create the VM, you specify a DNS name for it, for example "myxetest". This becomes the subdomain of Microsoft's "cloudapp.net" domain, for example "myxetest.cloudapp.net". If you want, you can use your own domain name by adding a CNAME record to your domain and pointing it towards the cloudapp.net subdomain.
  4. Use Remote Desktop to login to the new server.
  5. Download Oracle XE for 64-bit Windows. The file is around 300MB, so this takes less than a minute to download.
  6. Unzip the XE file and run setup.exe. Specify a password for the SYS and SYSTEM users, and let the installer do its work. The whole setup process took about 10-15 minutes on my server.
  7. Check that the database is running using sqlplus on the command line on the server.
  8. Oracle XE 11g comes with Apex 4.0 preinstalled, and running via the Embedded PL/SQL Gateway (DBMS_EPG) on port 8080 (localhost only). You can verify that it's up and running by going to http://127.0.0.1:8080/apex/apex on the server. If you want EPG to work for remote hosts, you have to execute the dbms_xdb.setlistenerlocalaccess procedure and pass FALSE to enable non-local access. (You can also change the port used by EPG via the dbms_xdb.sethttpport procedure.)
  9. You need to modify the Windows Firewall settings on the server to allow outbound traffic on port 8080. Finally, you also need to open up port 8080 for the VM in the Azure management portal (you'll find this setting under "Endpoints" for the VM.)

Here's a screenshot after the database has been installed:




The Apex homepage on localhost:



The Apex builder in action, accessed via the default cloudapp.net domain:




A note on production deployments


The Embedded PL/SQL Gateway (DBMS_EPG) is intended for development environments, and is not recommended for production. Since you are running on a Windows server, you probably want to leverage the excellent built-in Internet Information Server (IIS), with the Thoth Gateway which is an open source ASP.NET implementation of the OWA interface which is what the EPG, mod_plsql and the Apex Listener (now called Oracle REST Data Services, or ORDS) use to communicate with the database. You'll need to enable/install IIS first, as this is not enabled by default on a Windows 2008 server. Then download the Thoth Gateway and follow the installation instructions in the downloaded zip file.

Tuesday, June 24, 2014

Oracle Express Edition (XE) 11g for Windows 64 bit


As announced by Kris Rice (who is the product manager for SQL Developer, Apex Listener and Oracle XE), the Oracle Express Edition (XE) Database version 11g is now (finally!) available for 64-bit Windows.

It's been a looong wait; the 64-bit version for Windows was expected to be released around the same time as the 32-bit version, some 3 years ago.

Anyway, it's here now, which is great! Thanks Kris! :-)


Oracle Express Edition 11g can be downloaded here, it now comes in three flavors (Win 32 bit, Win 64 bit, and Linux 64 bit).



Saturday, July 13, 2013

First mentions of Oracle 12c XE (Express Edition)

Oracle released an Express Edition (XE) of its 10g database back in 2005 or thereabouts, and this was later followed up by an Oracle 11g Express Edition database.


Oracle XE is great because it is lightweight and "free to develop, deploy and distribute". Dietmar Aust has posted a thorough clarification on XE licensing, where representatives from Oracle have confirmed that XE is indeed free. (The post also includes some good security advice in the comments.) Oracle XE has its limitations, but I've previously posted some performance benchmarks for Oracle XE, which shows that it is more than good enough for many use cases.

Back to the topic of this post... during the Q&A part of the "Oracle Database 12c Launch Webcast" there were several questions about a 12c version of XE:

Will there be an Oracle 12c Express Edition? It is planned, but there are no dates yet.

Is there an Express edition of 12c available? Planned, but no dates yet.

Will 12c be available for Express Edition? If so, schedule? Planned but no dates yet.

So basically, we now know that there will be an Oracle 12c Express Edition coming out, but we have no idea when. Hopefully it won't take 5-6 years, like the wait between 10g and 11g... (and when it arrives it will probably be for 64-bit Windows, which was promised for XE 11g, but which never materialized UPDATE: Oracle XE 11g for 64-bit Windows was released in June 2014).

But hey, it's free, and it's a great product, so it will be worth the wait! :-)

Tuesday, March 1, 2011

Stress Testing Oracle XE 10g

Oracle Express Edition (XE) is Oracle's free entry-level database product, currently available only in a 10g version. XE is usually pitched as suitable for personal work or (very) small departmental applications, but I was curious as to what kind of load it can support.

Oracle XE 10g has the following limitations:

  • Up to 1 instance per server
  • Up to 1 CPU (will not use more even if available)
  • Up to 1 GB RAM (will not use more even if available)
  • Up to 4 GB datafiles (not including XE system data)
  • Free to develop, distribute and deploy in production


By the way, several things seem to indicate that Oracle Express Edition (XE) 11g is just around the corner. It's rumored that XE 11g will raise the datafile limit from 4 GB to 11 GB, but the other limits will remain as far as I know (and I don't really know anything about it...!). So these performance tests will probably be representative both for XE 10g and 11g, although I can't know for sure until 11g is available.

Here is the test environment I set up:

Hardware

Note that the "hardware" in this case actually runs virtualized in a VMWare environment.
Here are the resources allocated to the server:

  • 1 Intel Xeon X7350 2,9 GHz CPU
  • 4 GB RAM (but remember that XE won't use more than 1 GB anyway)
  • 30 GB disk space


Software

The software setup was as follows:

  • Windows Server 2003 R2 Standard Edition Service Pack 2
  • Microsoft Internet Information Server (IIS) 6.0
  • Thoth Gateway
  • Oracle Express Edition (XE) 10g
  • Oracle Application Express (Apex) 4.0


Note that I am not using the Embedded PL/SQL Gateway (DBMS_EPG) as the web server, but rather IIS in combination with the Thoth Gateway, a free ASP.NET replacement for mod_plsql and DBMS_EPG that I wrote in C# using ODP.NET.

Everything (database, web server, Apex) was installed using default settings.
The Apex images folder (/i/) was set up with "Expires"-headers (7 days) to allow browsers to cache images.

Both the database and the web server run on the same machine (server), in order to keep the setup as simple as possible (and show what is possible with just one box).

Test page

I set up an Apex test page with a mix of regions, including a report region, a couple of PL/SQL regions and an HTML region. For each page view, a random number is generated (to simulate different users looking at different things). The report query selects up to 200 rows based on this random number (using bind variables, of course), out of a system table/view (DBA_OBJECTS) with around 17,000 total rows.



There is also an after footer page process that inserts a row in a log table, so this is not just a read-only page.

The application was set up with an authorization scheme of "No application authorization scheme required" to allow it to be accessed from the online stress test tool.

Stress test tool

LoadImpact is a user-friendly, web-based stress testing service that allows you to run free tests that simulate up to 50 concurrent clients (you can pay to test with more clients). A key point here is that the load is actually generated from their test servers on the public internet, so this gives a more realistic test than simply running a stress testing tool on your local network.

You don't even need to sign up, just enter the URL of a website to test it (apparently, there are safeguards to prevent this from being used as a denial-of-service attack tool).

A note regarding image caching: The LoadImpact service does not cache anything. This means that all the Apex images, Javascript and stylesheets will be loaded on every page hit, even though in reality they would be cached in the user's browser after the first page view. Since I wanted to test the database's ability to handle page views, rather than IIS's ability to serve up static files, I changed the Apex page so that the standard Javascript and CSS files were not included. I then manually added the core stylesheets (core Apex + theme) back on the page via the page template, just to preserve the look and layout of the page. But even these could have been left out in order to simulate a scenario where most users have the files cached in their browser.

Test results

The free test at LoadImpact runs five different subtests, starting with 10 concurrent clients and ending with 50 concurrent clients. The full test takes 10 minutes, so that is in five 2-minute parts with increasing load.

While this was going on, I watched the CPU usage in the Windows Task Manager and grabbed the following screenshots:





While there are some peaks at the start of each subtest, it seems like the database "warms up" and actually does less work at the middle and end of the subtest compared to the start. Even at 40 and 50 clients, the CPU is (on average) not working at more than half capacity.

Here is the summary from LoadImpact at the end of the test:



First of all, note the nice and almost flat curve, with response times of under 300 ms even with 50 clients (and remember that the load generator is actually located in a different country, accessing the Apex page via the public Internet).

Actually, while the test was running, I was also clicking through a separate Apex application (an actual application, not just a test page) on the same server, and I couldn't really notice that the server was under any kind of stress; the response times were excellent.

The total number of requests for the whole test was 4500, but these requests include not only the actual page (the call to the "f" procedure) but also 2 stylesheets and 3 images. So the database was hit 1 in 6 times, in other words 16,7% (of 4500) or 750 times. Each hit inserted a row into a log table. If we query that log table and group by down to the second, we can get the number of database requests per second:



This shows that the XE database was handling 10 or 11 page views per second. If this can be sustained over time, that is 36,000 page views per hour, or 864,000 page views per day.

By comparison, apex.oracle.com is reported to have 4,800,000 page views per week or 685,000 page views per day. And according to this, Reddit has 10 pageviews per second per server (with 80 servers), while Facebook has just 2 pageviews per second per server (with 30,000 servers!).

I won't read too much into these numbers (as obviously a "page view" can vary wildly between different websites, and only actual use can tell whether your site is performing well or not), but clearly the free XE database is more than a toy.

I'm even tempted to buy a Basic or Professional test at LoadImpact to test with more than 50 clients and see where that curve leads with 60, 80 or 100 concurrent clients...