Swingbench Faq

The following is a list of questions commonly asked about the swingbench benchmarking environment.

  • Miscellaneous
  • Should I use 2.4 or 2.3 of swingbench
    Whilst there's not much superficially different between 2.3 and 2.4. Quite a lot has changed under the covers. 2.4 uses code from Datagenerator and CPUMonitor to extent its capabilities. Whilst 2.3 maybe a more stable platform I'd like people to give 2.4 a try and provide me with feedback.
  • Whats the difference between 2.3 and 2.4 of swingbench
    • New SH wizard
    • New highly threaded benchmark builds for the OE and SH benchmarks
    • New standard sizings for SOE and SH (1GB,10GB,100GB,1TB)
    • Improved scalability of the SOE benchmark
    • Oracle UCP connections
    • New CPU monitor architecture (uses ssh instead of agent)
    • Update look and feel on Overview charts (more coming)
    • Configuration free install (Simply ensure Java is your path)
  • Is swingbench really free?
    Yes. It comes as seen, there are no licenses or support charges. If you find it useful let us know.
  • Can I get support for swingbench?
    No. There is no support mechanism for swingbench, it is not an official Oracle product (hence the reason for it being on my personal website). I'll fix obvious bugs but sadly my full time job does not allow me to provide advice on training or how to configure swingbench. I have started improving the documentation and rounding off some of the rough edges which should help.
  • How do I report a bug for Swingbench/TraceAnalyzer/DataGenerator?
    Whilst no official system exisits for reporting bugs against swingbench you can email the problem directly to me via the comments page and I'll do my very best to resolve the issue in one of the following point releases. When you report the bug can you please ensure that you include
    • Swingbench version
    • The platform swingbench is running on
    • A description of the error
    • Any debug out put that you think is releveant (In 2.3 try running with the -debug option)
  • Can I raise TAR's against swingbench?
    No. As I indicated above there is no official support channel for Swingbench. If its something you simply can't figure out leave me a message on comments page and I'll get back to you as soon as I can.
  • Can I get hold of the source code for swingbench?
    No. Im not in a position to distribute the Swingbench kernel however I do distribute all of the source code for the transactions which can be viewed and modified. This is still my intellectual property and shouldn't be used with anything other than the Swingbench framework.
  • Can swingbench be used to benchmark hardware?
    Whilst this is one of the main uses of Swingbench it has to be stated that because it is an unofficial product the author or Oracle coorporation will not offer an guarantees on the validity of the results. It should primarily be used as a guide line.
  • Can I run swingbench against databases other than Oracle
    This is not the aim of Swingbench. It is designed as a support/demo tool of Oracle technology. We have no plans to extend its functionality to run against non Oracle databases. Currently it supports Oracle and TimesTen only.
  • Where can I find up to date info on swingbench
    I try and update my thoughts on the development of swingbench and any minor changes I make inside of my blog which you can find it here.
  • Can I change the transactions used by swingbench or include by own.
    Yes. The source code for all of the transactions is included in the distribution. It can be found under the $SWINGHOME/source directory. An "ant" script is also shipped that easily compiles all of the source code for you. It is also possible to modify some simple PL/SQL packages to include your own code. This screen cast describes all of the ways possible to modify swingbench.
  • Why hasn't there been a new release lately?
    Developing swingbench is not my full time (or even part time) job. It is done to support projects inside of Oracle.
  • Configuration
  • How do I automate swingbench for testing
    The easiest way to automate several runs of swingbench is by using charbench and command line options. This enables the scripting of several runs that can be run "lights out". An example of this might be something like

    bin $> time ./oewizard -cl -create -scale 1 -u soe1 -p soe1 -ts soescale1 -tc 16 -s
    bin $> ./charbench -u soe1 -p soe1 -uc 100 -min 10 -max 200 -rt 0:10 -a -s -r scale1_100user.xml
    bin $> ./charbench -u soe1 -p soe1 -uc 200 -min 10 -max 200 -rt 0:10 -a -s -r scale1_200user.xml
    bin $> ./charbench -u soe1 -p soe1 -uc 300 -min 10 -max 200 -rt 0:10 -a -s -r scale1_300user.xml
    bin $> time ./oewizard -cl -create -scale 10 -u soe10 -p soe10 -ts soescale10 -tc 16 -s
    bin $> ./charbench -u soe10 -p soe10 -uc 100 -min 10 -max 200 -rt 0:10 -a -s -r scale10_100user.xml
    bin $> ./charbench -u soe10 -p soe10 -uc 200 -min 10 -max 200 -rt 0:10 -a -s -r scale10_200user.xml
    bin $> ./charbench -u soe10 -p soe10 -uc 300 -min 10 -max 200 -rt 0:10 -a -s -r scale10_300user.xml

    This example uses oewizard to create a schema "soe1" in a tablespace soescale1 using 16 threads to build it. It then runs 3 workloads against the schema increasing the user count for each run. It then builds a bigger schema (scale 10 = 32GB of space) and reruns the test.
  • What does the transaction load ratio mean?
    The load ratio is the ratio in comparison to other transactions. ie.

    • T1 load ratio 10 = typically executes 16% of the time
    • T2 load ratio 20 = typically executes 33% of the time
    • T3 load ratio 30 = typically executes 50% of the time
    Load ratios allow more precise control of the transactions. You change the ratios by modify the values within the config file or by changing them with the swingbench UI as shown below.

    screenshot_03
  • What is the maximum CPU load I should run a load generator at?
    In theory the load generator(s) should be run on a seperate machine to the database and the average load should be kept below 70%. This generally results in reliable results
  • Whats the best version of swingbench to run my tests with?
    Currently Im recommending 2.3. It has had a number of bug fixes and functionality enhancements that should make it easier to use.
  • How many load generators (servers) will it take to fully load a database server with?
    It Depends. Typically it is a ratio of one load generator CPU to two database CPU's That is to say it would take a 2 CPU machine to fully load a 4 CPU machine. This assumption is based on the CPU's being of equal processing power and the load being run with zero think time. It is usually the case that you will need as least 2-4 users/threads per CPU.
  • Whats the difference between swingbench,minibench and charbench
    Swingbench, minibench and charbench are simply frontends on the swingbench kernel. Swinbench is a rich fully functional frontend that includes several real time charts and as a result has a significant cpu cost associated with it. Minibench is a simple graphical tool without the overhead of Swingbench but is useful for users who like to be able to see what is happening in an organised and controlled fashion. Charbench is a character front end that enables the load generator to be be run where it is not possible/sensible to use a graphical front end. All three of the front ends have the same functionality and are interchangeable with one another.
  • I keep geeting a java exception "java.lang.NoSuchMethodError: oracle.jdbc.pool.OracleDataSource". What am I doing wrong.
    Swingbench require's the latest versions of jdbc to work properly. Download the 11g jdbc drivers from Oracle and use these even when running against a 9i database.
  • Can I run mulitiple load generators against a single database instance?
    Yes. In fact it is adviseable when running a large load against a database to use multiple copies of swingbench. These can be coordinated using clusteroverview.
  • Clusteroverview dosen't work on windows/linux when I upgrade to 2.2
    Some of the parameters in both swingconfig.xml and clusteroverview.xml have changed between 2.1g and 2.2. The most signicant of these was the "Coordinator" Element this has a child called "Server" in 2.1g this has changed name to "Location" in 2.2. Please use the latest config files shipped in 2.2.
  • Which JVM should I use?
    Currently swingbench requires at least a 1.5 JVM (Java 5) to run. Java 6 has been tested and works fine if your platform supports it
  • How do I get CPU monitoring working?
    You need to start cpumonitor (in the $SWINGHOME/bin directory) on the platform you want to monitor. i.e.

    bin $> ./cpumonitor
    bin $> CPU monitor started started Successfully

    Then all you need to do is connect using the command line option "-cpuloc" in swingbench/minibench/charbench i.e.

    bin $> ./swingbench -cpuloc localhost

    Alternatively you can specify it in the "Distributed" tab in swingbench.

    screenshot_02
  • Why don't I see CPU and disk statistics inside of swingbench when running on the Windows platform?
    The CPU/Disk monitor used by swingbench uses the common unix utility vmstat to calculate the load on a target platform. This utility is not availble by default for Windows however it can be obtained via the Cygwin environment (www.cygwin.com). Currently Solaris only reports CPU and AIX dosen't report either statistic. This is being fixed
  • I don't get any charts, other than user count, inside of clusteroverview, what have I broken?
    This usually occurs because the the "DisplayName" in clusteroverview.xml is different to the connect string being used by a load generator. Presently if you wish to measure the scalability of a cluster the DisplayName attribute must match one or more load generators connect strings. i.e if you have a load generator(s) with a connect string of //node1:1521/soeservice the DisplayName must be //node1:1521/soeservice .
  • Benchmarks
  • What should the SOE benchmark look like on completition
    You should have the following tables and and index count. Ignore the row counts these will depend on the size of the benchmark you selected.
    Tables
    ======

    Table                   Rows    Blocks   Size   Compression  Indexes Partitions Analyzed
    -------------------- ---------- -----    ------ -----------  ------- ---------- ----------
    WAREHOUSES                1,000     60    1024k Disabled       2          0     < Week
    ORDERS                  225,000  1,636      13M Disabled       5          0     < Week
    INVENTORIES             924,859 10,996      87M Disabled       3          0     < Week
    ORDER_ITEMS             587,151  2,392      19M Disabled       3          0     < Week
    PRODUCT_DESCRIPTIONS      1,000     60    1024k Disabled       2          0     < Week
    LOGON                    50,000    250       2M Disabled       0          0     < Week
    PRODUCT_INFORMATION       1,000     60    1024k Disabled       3          0     < Week
    CUSTOMERS               200,000  2,014      16M Disabled       5          0     < Week
  • I don't appear to have the right number of indexes
    This is probably because you either ran out of space or you didn't size your TEMP correctly. As a guide line for a schema of size “x” you'll need at least “x/6” worth of temp space i.e. 1TB schema needs about 180GB of temp. You can resize it after the build to what ever you decide is appropriate.
  • Nothing happens for ages when I start the SOE benchmark
    If you've created a large schema make sure you've set the SOE_MIN_CUSTOMER_|D and SOE_MAX_CUSTOMER_ID environment variables. To set them follow the instructions below or edit the config file

    Select the Environment Variables tab and press the screenshot_02 button (you’ll need to do this for each environment variable).

    screenshot_04

    Add two Enviroment variables
    • SOE_MIN_CUSTOMER_ID : The value equals the smallest customer id in the data set, usually 1
    • SOE_MAX_CUSTOMER_ID : The largerst customer id found in the data set

    You can determine what thes values are by running a piece of SQL similar to this when logged into the SOE schema

    SELECT
      /*+ PARALLEL(CUSTOMERS, 8) */
      MIN(customer_id) SOE_MIN_CUSTOMER_ID,
      MAX(customer_id) SOE_MAX_CUSTOMER_ID
       FROM customers


    After adding the variables you should end up with something that looks similar to this

    screenshot_06
  • The benchmark isn't running as quickly as I thought it would.
    That's not unexpected. Some features of the benchmarks are designed to introduce a degree of contention to determine how well the underlying hardware handles it. It is unlikely you will be able to get the CPU to run at 100% especially as you increase the size of the schema. Use the AWR reports to determine what the issue is. Swingbench 2.3 enables you to take database snapshots at the start and end to determine the cause of wait event. You can enable this using the relevant fields within swingbench (shown in the image below). This will take a database snap at the start and end of the benchmark. Get your friendly tuning guru to take a look and make some recommendations.
    screenshot_02
    I also put the top ten wait events and their percentages into the output report (output tab in swingbench, results.xml in minibench and charbench) if the "Collect database statistics" option is checked. They should look something like this.
    <DatabaseWaitEvents>
          <DatabaseWaitEvent name="CPU Time" noOfTimesWaited="1" timeWaited="4729" percentageTimeWaited="51.26"/>
          <DatabaseWaitEvent name="log file sync" noOfTimesWaited="32230" timeWaited="1950" percentageTimeWaited="21.14"/>
          <DatabaseWaitEvent name="log file parallel write" noOfTimesWaited="48457" timeWaited="1247" percentageTimeWaited="13.52"/>
          <DatabaseWaitEvent name="cell single block physical read" noOfTimesWaited="1012" timeWaited="567" percentageTimeWaited="6.15"/>
          <DatabaseWaitEvent name="db file parallel write" noOfTimesWaited="7320" timeWaited="566" percentageTimeWaited="6.13"/>
          <DatabaseWaitEvent name="control file sequential read" noOfTimesWaited="372" timeWaited="30" percentageTimeWaited="0.33"/>
          <DatabaseWaitEvent name="reliable message" noOfTimesWaited="1" timeWaited="22" percentageTimeWaited="0.24"/>
          <DatabaseWaitEvent name="gc cr grant 2-way" noOfTimesWaited="617" timeWaited="19" percentageTimeWaited="0.21"/>
          <DatabaseWaitEvent name="buffer busy waits" noOfTimesWaited="1182" timeWaited="16" percentageTimeWaited="0.17"/>
          <DatabaseWaitEvent name="ges message buffer allocation" noOfTimesWaited="98591" timeWaited="12" percentageTimeWaited="0.13"/>
          <DatabaseWaitEvent name="SQL*Net message to client" noOfTimesWaited="44031" timeWaited="11" percentageTimeWaited="0.12"/>
       </DatabaseWaitEvents>
  • Why do I need to keep generating new sets of data for each Callingcircle benchmark run?
    Callingcircle transactions are based on "customers" that need to be processed or have their details updated. The generation process looks for likely candidates and writes them to files. Each benchmark run updates customer details and so new candidates need to be found. Eventually a significant proportion of available cusotmers have had their details updated and so the entire benchmark needs to be updated. It is also important that if you are using multiple load generators each has its own set of generated data.
  • How many transactions do I need to generate for a Callingcirle benchmark run?
    It depends. The more powerful the machine/cluster the faster the transactions will be processed. A thousand transactions lasts only three minutes on a Xeon processor. Therefore to generate a 30 minute load you'd need at least 10,000 transactions and probably 40,000 transactions for a 4 CPU machine.

    Update : I've been informed of an issue where its not possible to hold all of the transactions in memory for a long sustained run on a powerful machine. Im working on a solution to enable a disk based loading mechanism.
  • Why are there two versions of the Order Entry benchmark?
    One uses PL/SQL stored procedures to generate a load and the other uses discrete java routines and individual jdbc statements, as a result the later will generate a lot of network traffic. We would recomend the use of the PL/SQL version of the benchmark.
  • Order Entry dosen't appear to scale as well as Callingcircle, why?
    Order Entry updates a relatively small table containing stock levels at each warehouse. This creates a great deal of contention and limits it scalability.
  • Whats the difference between the Callingcircle benchmark and Order Entry benchmark?
    Order entry models the classic order entry stress test. It has a similar profile to the TPC-C benchmark. This version models a online order entry system with users being required to log-on before purchasing goods. The Calling Circle benchmark represents a self-service OLTP application. The application models the customers of a telecommunications company registering, updating and inquiring on a calling circle of their most frequently called numbers in order to receive discounted call pricing. It is characterized by large amounts of dynamic PL/SQL and is CPU intensive. Calling Circle also requires the regeneration of data after each run.
  • Is there a datawarehousing benchmark available?
    In the latest 2.3 build of swingbench (December 2009) there is a new wizard "shwizard" that builds a sales history schema. It is currently under test but it can be used to build a sizeable test database. Swingbench also includes a config (shconfig under the $SWINGHOME/bin/sample directory) with and a number of queries to stress the resultant schema. I'll be producing a web page shortly to describe its setup and configuration.