Should I use 2.4 or 2.3 of swingbench
Swingbench 2.4 is the latest most stable release. Please give it a go and provide me with feedback
Whats the difference between 2.3 and 2.4 of swingbench
- New SH wizard
- New highly threaded benchmark builds for the OE and SH benchmarks
- New standard sizings for SOE and SH (1GB,10GB,100GB,1TB)
- Improved scalability of the SOE benchmark
- Oracle UCP connections
- New CPU monitor architecture (uses ssh instead of agent)
- Update look and feel on Overview charts (more coming)
- Configuration free install (Simply ensure Java is your path)
Is swingbench really free?
Yes. It comes as seen, there are no licenses or support charges. If you find it useful let us know.
Can I get support for swingbench?
No. There is no support mechanism for swingbench, it is not an official Oracle product (hence the reason for it being on my personal website). I'll fix obvious bugs but sadly my full time job does not allow me to provide advice on training or how to configure swingbench. I have started improving the documentation and rounding off some of the rough edges which should help.
How do I report a bug for Swingbench/TraceAnalyzer/DataGenerator?
Whilst no official system exisits for reporting bugs against swingbench you can email the problem directly to me via the comments
page and I'll do my very best to resolve the issue in one of the following point releases. When you report the bug can you please ensure that you include
- Swingbench version
- The platform swingbench is running on
- A description of the error
- Any debug out put that you think is releveant (In 2.3 try running with the -debug option)
Can I raise SRs against swingbench?
No. As I indicated above there is no official support channel for Swingbench. If its something you simply can't figure out leave me a message on comments
page and I'll get back to you as soon as I can. If there is enough interest I'll put together a forum or bug repository.
Can I get hold of the source code for swingbench?
No. Im not in a position to distribute the Swingbench kernel however I do distribute all of the source code for the transactions which can be viewed and modified. This is still my intellectual property and shouldn't be used with anything other than the Swingbench framework.
Can swingbench be used to benchmark hardware?
Whilst this is one of the main uses of Swingbench it has to be stated that because it is an unofficial product the author or Oracle coorporation will not offer an guarantees on the validity of the results. It should primarily be used as a guide line.
Can I run swingbench against databases other than Oracle
This is not the aim of Swingbench. It is designed as a support/demo tool of Oracle technology. We have no plans to extend its functionality to run against non Oracle databases. Currently it supports Oracle and TimesTen only.
Where can I find up to date info on swingbench
I try and update my thoughts on the development of swingbench and any minor changes I make inside of my blog which you can find it here
Can I change the transactions used by swingbench or include by own.
Yes. The source code for all of the transactions is included in the distribution. It can be found under the $SWINGHOME/source directory. An "ant" script is also shipped that easily compiles all of the source code for you. It is also possible to modify some simple PL/SQL packages to include your own code. This screen cast describes
all of the ways possible to modify swingbench.
Why hasn't there been a new release lately?
Developing swingbench is not my full time (or even part time) job. It is done to support projects inside of Oracle.
I'm struggling to get swingbench to work with 64bit OCI libraries I get " ELFCLASS64 (Possible cause: architecture word width mismatch)"
This is because swingbench boot straps another JVM. To fix this issue you need to edit the $SWINGHOME/launcher/launcher.xml file. Change the following section in the file
Also make sure you are pointing to the right version of the jdbcoci libraries.
What versions of jdbc do you use?
Swingbench ships with the current production jdbc drivers available from the Oracle website. It is possible to change these to older or newer versions by simply removing ons.jar,ucp.jar and ojdbc6.jar from the lib directory and replacing them with your preferred equivalents
How does cpumonitor work in 2.4?
The new functionality in 2.4 for monitoring System resources (Disk and CPU) uses Secure Shell for accessing the remote system
I get out of memory messages when running swingbench i.e. java.lang.OutOfMemoryError: Java heap space
This is caused by a variety of reasons. It might be caused because statistics are being collected over a long period of time. The amount of memory allocated to swingbench can be changed by editing the files responsible for launching the code.
In 2.3 edit the shell or bat file responsible for launching the code i.e. swingbench, minibench,charbench
# The following command enables debug mode
becomes (if your increasing the default memory used to 1GB)
$JAVAHOME/bin/java -DSyslog.level=DEBUG -mx250m com.dom.benchmarking.swingbench.SwingBench $*
#$JAVAHOME/bin/java -mx250m com.dom.benchmarking.swingbench.SwingBench $*
# The following command enables debug mode
$JAVAHOME/bin/java -DSyslog.level=DEBUG -mx1024m com.dom.benchmarking.swingbench.SwingBench $*
#$JAVAHOME/bin/java -mx250m com.dom.benchmarking.swingbench.SwingBench $*
In 2.4 You need to edit the launcher file not the file the shell or batch file start starts the program. This file is located in the "launcher" directory and is called "launcher.xml". Simply edit the file and change the values to reflect your required memory requirements. You only need to change the section titled jvmargset i.e.
becomes (if your increasing the default memory used to 4GB)
How do I automate swingbench for testing
The easiest way to automate several runs of swingbench is by using charbench and command line options. This enables the scripting of several runs that can be run "lights out". An example of this might be something like
bin $> time ./oewizard -cl -create -scale 1 -u soe1 -p soe1 -ts soescale1 -tc 16 -s
bin $> ./charbench -u soe1 -p soe1 -uc 100 -min 10 -max 200 -rt 0:10 -a -s -r scale1_100user.xml
bin $> ./charbench -u soe1 -p soe1 -uc 200 -min 10 -max 200 -rt 0:10 -a -s -r scale1_200user.xml
bin $> ./charbench -u soe1 -p soe1 -uc 300 -min 10 -max 200 -rt 0:10 -a -s -r scale1_300user.xml
bin $> time ./oewizard -cl -create -scale 10 -u soe10 -p soe10 -ts soescale10 -tc 16 -s
bin $> ./charbench -u soe10 -p soe10 -uc 100 -min 10 -max 200 -rt 0:10 -a -s -r scale10_100user.xml
bin $> ./charbench -u soe10 -p soe10 -uc 200 -min 10 -max 200 -rt 0:10 -a -s -r scale10_200user.xml
bin $> ./charbench -u soe10 -p soe10 -uc 300 -min 10 -max 200 -rt 0:10 -a -s -r scale10_300user.xml
This example uses oewizard to create a schema "soe1" in a tablespace soescale1 using 16 threads to build it. It then runs 3 workloads against the schema increasing the user count for each run. It then builds a bigger schema (scale 10 = 32GB of space) and reruns the test.
What does the transaction load ratio mean?
The load ratio is the ratio in comparison to other transactions. ie.
- T1 load ratio 10 = typically executes 16% of the time
- T2 load ratio 20 = typically executes 33% of the time
- T3 load ratio 30 = typically executes 50% of the time
Load ratios allow more precise control of the transactions. You change the ratios by modify the values within the config file or by changing them with the swingbench UI as shown below.
What is the maximum CPU load I should run a load generator at?
In theory the load generator(s) should be run on a seperate machine to the database and the average load should be kept below 70%. This generally results in reliable results
Whats the best version of swingbench to run my tests with?
Currently Im recommending 2.4. which has a large number of bug fixes and functionality enhancements that should make it easier to use. Anyone looking to use high end OLTP testing should always use the 2.4 version of swingbench
How many load generators (servers) will it take to fully load a database server with?
It Depends. Typically it is a ratio of one load generator CPU to two database CPU's That is to say it would take a 2 CPU machine to fully load a 4 CPU machine. This assumption is based on the CPU's being of equal processing power and the load being run with zero think time. It is usually the case that you will need as least 2-4 users/threads per CPU.
Whats the difference between swingbench,minibench and charbench
Swingbench, minibench and charbench are simply frontends on the swingbench kernel. Swinbench is a rich fully functional frontend that includes several real time charts and as a result has a significant cpu cost associated with it. Minibench is a simple graphical tool without the overhead of Swingbench but is useful for users who like to be able to see what is happening in an organised and controlled fashion. Charbench is a character front end that enables the load generator to be be run where it is not possible/sensible to use a graphical front end. All three of the front ends have the same functionality and are interchangeable with one another.
I keep geeting a java exception "java.lang.NoSuchMethodError: oracle.jdbc.pool.OracleDataSource". What am I doing wrong.
Swingbench require's the latest versions of jdbc to work properly. Download the 11g jdbc drivers from Oracle and use these even when running against a 9i database.
Can I run mulitiple load generators against a single database instance?
Yes. In fact it is adviseable when running a large load against a database to use multiple copies of swingbench. These can be coordinated using clusteroverview.
Is there a walkthough available for clusteroverview in 2.3 and 2.4
Yes it can be found here
Which JVM should I use?
Swingbench 2.3 and Swingbench 2.4 have been tested with both Java 6 and Java7. Use which ever is supported in your environment.
How do I get CPU monitoring working?
For 2.3 only
you need to start cpumonitor (in the $SWINGHOME/bin directory) on the platform you want to monitor. i.e.
bin $> ./cpumonitor
bin $> CPU monitor started started Successfully
Then all you need to do is connect using the command line option "-cpuloc" in swingbench/minibench/charbench i.e.
bin $> ./swingbench -cpuloc localhost
Alternatively you can specify it in the "Distributed" tab in swingbench.
*NOTE : The functionality in 2.4 has changed and it now uses ssh to "scrape" information from the target server directly without the need for a remote client.
To configure it simply go to the Distributed Controls tab and enter the hostname of a server you wish to monitor. The username and password should be of a user who can run vmstat. You can test the connection with the button to highlight any security issues.
Why don't I see CPU and disk statistics inside of swingbench when running on the Windows platform?
The CPU/Disk monitor used by swingbench uses the common unix utility vmstat to calculate the load on a target platform. This utility is not availble by default for Windows however it can be obtained via the Cygwin environment (www.cygwin.com). Currently Solaris only reports CPU and AIX dosen't report either statistic. This is being fixed
I don't get any charts, other than user count, inside of clusteroverview, what have I broken?
This usually occurs because the the "DisplayName" in clusteroverview.xml is different to the connect string being used by a load generator. Presently if you wish to measure the scalability of a cluster the DisplayName attribute must match one or more load generators connect strings. i.e if you have a load generator(s) with a connect string of
the DisplayName must be
How do I modify or rename supplied benchmark transactions
Whilst this isn't the place to show you how to develop swingbench transactions in java I can show you basic steps to modify or rename the supplied benchmark transactions.
You might want to do this if you wanted to develop your own code, add more transactions or simply call them something else.
You do this with a few simple steps (The following assumes that swingbench is installed in your home directory)
1/ change directory into the base swingbench source directory
2/ take a copy of one of the directories
cp -r storedprocedures/ newtransactions
3/ cd into the new directory
4/ Delete all but one of the transactions (it will save a little time later)
5/ Rename the transaction to your preferred name
mv StoredProcedure1.java TopPerformers.java
6/ Edit the java file to reflect the name change
The first change reflects the new package name. The second and reflects the new name of the transaction you are creating.
7/ repeat step 5 and 6 for all of the transactions you wish to create. You might at this stage also modify the code to reflect any other functionality changes you want to make.
8/ change directory to the main swingbench source directory
9/ run ant to compile the code (it will use the build.xml file in the source directory). You'll need to make sure that java is in your path etc.
[echo] Building file /Users/dgiles/swingbench/lib/mytransactions.jar from the directory /Users/dgiles/swingbench
[mkdir] Created dir: /Users/dgiles/swingbench/classes
[javac] /Users/dgiles/swingbench/source/build.xml:19: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
[javac] Compiling 44 source files to /Users/dgiles/swingbench/classes
[javac] Note: Some input files use unchecked or unsafe operations.
[javac] Note: Recompile with -Xlint:unchecked for details.
[jar] Building jar: /Users/dgiles/swingbench/lib/mytransactions.jar
[delete] Deleting directory /Users/dgiles/swingbench/classes
Total time: 1 second
10/ The step above will not only compile the code but build a "jar" file and copy it into your swingbench/lib directory. The next step will be to use the transactions inside of swingbench. To do this simply make a copy of the default swingbench config file and edit this
$ cd ../bin
$ cp swingconfig.xml newtransactions.xml
$ edit newtransactions.xml
11/ Edit the config file to reflect the work carried out in the previous steps
12/ Save the file and start swingbench using the new configuration
$ ./swingbench -c newtransactions.xml
And thats it.
How do I control the number of users logging on to prevent log on storms
Whilst Oracle deals very well with large numbers of users logging on at the same time some approaches, such as connection pooling, depend on the gradual logging on of users to work efficiently. Swingbench mimics this effect by logging users on in batches and enabling to begin their transactions as soon as they've logged on. To define your log on profile you'll need to enter values in the "Load" section. In the example below we've asked swingbench to log users on in batches of 10 every second and by specifying "Wait Till All Session Log On" to false swingbench will allow the session to begin running their transactions as soon as they've successfully connected. This means that for the example below it will take 50 seconds to load all of the users on and transactions will begin immediately.
The profile you use will depend entirely on the number of sessions to be logged on and the processing they will do when logged on. Most systems don't logon 10,000 sessions straight onto their server first thing in the morning. It is likely to be a steady increase over a period of time. Logging users on whilst transactions are being processed is a good test of the network and power of your server.
Can I set various connection properties i.e. "Statement Caching"
Yes. From Swingbench 2.5 you can set connection properties via a set of drop down lists in the connection properties dialogue.
The following table describe valid connection properties.
|StatementCaching||This specifies the number of statements to be cached. Valid value is an integer |
|DNSLoadBalancing||Force jdbc to connect to more than one scan listener. Valid values are true or false|
|FastFailover||Activate Oracle’s Fast Failover connection functionality. Valid values are true or false|
|TcpNoDelay||Sets TCP_NODELAY of a socket. Valid values are true or false|
|OnsConfiguration||The remote configurations of Oracle Notification Servers (ONS). Valid values are similar to this “nodes=dbserver1:6200,dbserver2:6200”. See Oracle documentation for more details and examples|
|MaxONSConfiguration||The number of ONS servers that jdbc will randomly select from in the advent of a failure. Valid value is an integer|
|TcpConnectionTimeout||The time taken between traversals of an “ADDRESS_LIST” in the advent of a failure. Specified in seconds|
|ReadTimeOut||The time queries wait for results before considering that a failure has occurred. Specified in seconds|
|BatchUpdates||The number of inserts/updates that are grouped together to improve transaction performance. Valid value is an integer|
|FetchSize||Number of rows fetched on each round trip to the database. Valid value is an integer|
Whats the difference between "inter" and "intra" sleep times in swingbench?
As the name implies intra sleep times occur “inside” of a transaction. Inter sleep times occur between transactions. Many of the transactions inside of the swingbench “SOE” have sleep times between DML operations (select, insert, update). In some situations this better emulates what happens in some legacy form based systems, this is what is controlled by intra sleep times. However most systems these days tend to utilise web based front ends where DML operations tend to be fired as a single operation when the user submits a form. This approach results in a more scalable architecture with fewer locks being held and for shorter periods of time. Hopefully the following diagram will explain the differences in a clearer fashion.
You can set inter and intra sleep times in the "Load" tab of the swingbench GUI as shown below
You can also set the intra and inter sleep time from the command line with the -min (intra min) -max (intramax) -intermin (inter min) -intermax (inter max).
How can I tell if the benchmark has installed correctly?
In 2.4 of swingbench the wizard will check whether the schema has the right number of tables/indexes/procedures etc. At the end of the install it will display a small report providing details of the run time and a list of invalid and valid objects (see below). All of the objects that have been created should be valid.
I get warnings during the build of a benchmark
This may be perfectly normal. They may occur if the scripts attempt to drop objects that don't exist. Check their context and the final report to see if all of the objects are valid.
What should the SOE benchmark look like on completition
You should have the following tables and and index count. Ignore the row counts these will depend on the size of the benchmark you selected.
Table Rows Blocks Size Compression Indexes Partitions Analyzed
-------------------- ---------- ----- ------ ----------- ------- ---------- ----------
WAREHOUSES 1,000 60 1024k Disabled 2 0 < Week
ORDERS 225,000 1,636 13M Disabled 5 0 < Week
INVENTORIES 924,859 10,996 87M Disabled 3 0 < Week
ORDER_ITEMS 587,151 2,392 19M Disabled 3 0 < Week
PRODUCT_DESCRIPTIONS 1,000 60 1024k Disabled 2 0 < Week
LOGON 50,000 250 2M Disabled 0 0 < Week
PRODUCT_INFORMATION 1,000 60 1024k Disabled 3 0 < Week
CUSTOMERS 200,000 2,014 16M Disabled 5 0 < Week
I don't appear to have the right number of indexes
This is probably because you either ran out of space or you didn't size your TEMP correctly. As a guide line for a schema of size “x” you'll need at least “x/6” worth of temp space i.e. 1TB schema needs about 180GB of temp. You can resize it after the build to what ever you decide is appropriate.
Nothing happens for ages when I start the SOE benchmark
If you've created a large schema make sure you've set the SOE_MIN_CUSTOMER_|D and SOE_MAX_CUSTOMER_ID environment variables. To set them follow the instructions below or edit the config file
Select the Environment Variables tab and press the
button (you’ll need to do this for each environment variable).
Add two Enviroment variables
- SOE_MIN_CUSTOMER_ID : The value equals the smallest customer id in the data set, usually 1
- SOE_MAX_CUSTOMER_ID : The largerst customer id found in the data set
You can determine what thes values are by running a piece of SQL similar to this when logged into the SOE schema
/*+ PARALLEL(CUSTOMERS, 8) */
After adding the variables you should end up with something that looks similar to this
The benchmark isn't running as quickly as I thought it would.
That's not unexpected. Some features of the benchmarks are designed to introduce a degree of contention to determine how well the underlying hardware handles it. It is unlikely you will be able to get the CPU to run at 100% especially as you increase the size of the schema. Use the AWR reports to determine what the issue is. Swingbench 2.3 enables you to take database snapshots at the start and end to determine the cause of wait event. You can enable this using the relevant fields within swingbench (shown in the image below). This will take a database snap at the start and end of the benchmark. Get your friendly tuning guru to take a look and make some recommendations.
I also put the top ten wait events and their percentages into the output report (output tab in swingbench, results.xml in minibench and charbench) if the "Collect database statistics" option is checked. They should look something like this.
<DatabaseWaitEvent name="CPU Time" noOfTimesWaited="1" timeWaited="4729" percentageTimeWaited="51.26"/>
<DatabaseWaitEvent name="log file sync" noOfTimesWaited="32230" timeWaited="1950" percentageTimeWaited="21.14"/>
<DatabaseWaitEvent name="log file parallel write" noOfTimesWaited="48457" timeWaited="1247" percentageTimeWaited="13.52"/>
<DatabaseWaitEvent name="cell single block physical read" noOfTimesWaited="1012" timeWaited="567" percentageTimeWaited="6.15"/>
<DatabaseWaitEvent name="db file parallel write" noOfTimesWaited="7320" timeWaited="566" percentageTimeWaited="6.13"/>
<DatabaseWaitEvent name="control file sequential read" noOfTimesWaited="372" timeWaited="30" percentageTimeWaited="0.33"/>
<DatabaseWaitEvent name="reliable message" noOfTimesWaited="1" timeWaited="22" percentageTimeWaited="0.24"/>
<DatabaseWaitEvent name="gc cr grant 2-way" noOfTimesWaited="617" timeWaited="19" percentageTimeWaited="0.21"/>
<DatabaseWaitEvent name="buffer busy waits" noOfTimesWaited="1182" timeWaited="16" percentageTimeWaited="0.17"/>
<DatabaseWaitEvent name="ges message buffer allocation" noOfTimesWaited="98591" timeWaited="12" percentageTimeWaited="0.13"/>
<DatabaseWaitEvent name="SQL*Net message to client" noOfTimesWaited="44031" timeWaited="11" percentageTimeWaited="0.12"/>
Why do I need to keep generating new sets of data for each Callingcircle benchmark run?
Callingcircle transactions are based on "customers" that need to be processed or have their details updated. The generation process looks for likely candidates and writes them to files. Each benchmark run updates customer details and so new candidates need to be found. Eventually a significant proportion of available customers have had their details updated and so the entire benchmark needs to be updated. It is also important that if you are using multiple load generators each has its own set of generated data.
How many transactions do I need to generate for a Callingcirle benchmark run?
It depends. The more powerful the machine/cluster the faster the transactions will be processed. A thousand transactions lasts only three minutes on a Xeon processor. Therefore to generate a 30 minute load you'd need at least 10,000 transactions and probably 40,000 transactions for a 4 CPU machine.
Update : I've been informed of an issue where its not possible to hold all of the transactions in memory for a long sustained run on a powerful machine. Im working on a solution to enable a disk based loading mechanism.
Why are there two versions of the Order Entry benchmark?
One uses PL/SQL stored procedures to generate a load and the other uses discrete java routines and individual jdbc statements, as a result the later will generate a lot of network traffic. We would recomend the use of the PL/SQL version of the benchmark.
Order Entry dosen't appear to scale as well as Callingcircle, why?
Order Entry updates a relatively small table containing stock levels at each warehouse. This creates a great deal of contention and limits it scalability.
Whats the difference between the Callingcircle benchmark and Order Entry benchmark?
Order entry models the classic order entry stress test. It has a similar profile to the TPC-C benchmark. This version models a online order entry system with users being required to log-on before purchasing goods. The Calling Circle benchmark represents a self-service OLTP application. The application models the customers of a telecommunications company registering, updating and inquiring on a calling circle of their most frequently called numbers in order to receive discounted call pricing. It is characterised by large amounts of dynamic PL/SQL and is CPU intensive. Calling Circle also requires the regeneration of data after each run.
New transactions in the orderentry benchmark
I've included three additional transactions in the orderentry benchmark to enable users to increase the "Read" workload for the Oracle database.
The transactions are disabled by default and so won't effect any historical comparisons. To enable them for a benchmark run simply check the box on the right hand side of the table.
Is there a datawarehousing benchmark available?
In swingbench 2.3 and 2.4 there is a new wizard "shwizard" that builds a sales history schema. It is currently under test but it can be used to build a sizeable test database. Swingbench also includes a config (shconfig under the $SWINGHOME/configs directory) with and a number of queries to stress the resultant schema. I'll be producing a web page shortly to describe its setup and configuration.