Static Analysis

Subscribe to Static Analysis: eMailAlertsEmail Alerts newslettersWeekly Newsletters
Get Static Analysis: homepageHomepage mobileMobile rssRSS facebookFacebook twitterTwitter linkedinLinkedIn

Static Analysis Authors: AppDynamics Blog, Jason Bloomberg, RealWire News Distribution, Skytap Blog, Jayaram Krishnaswamy

Related Topics: Static Analysis

testsiteforbugresearchpleasedonotapprove: Article

Product Review: Parasoft WebKing

A useful set of tools for preventing bugs from infiltrating software before deployment

Quality-conscious developers are familiar with the idea of coding checklists. The code you write must measure up to all the criteria on the checklist, from "no grammatical errors in the comments" to "performs all required functions." Based on these checklists, we have code reviews. A good code review takes time, but is certainly worth the effort. Such reviews can prevent many costly errors. However, when crunch time hits, thorough code reviews are often impossible. That's where a tool like Parasoft's WebKing can help.

For several decades tools to automatically generate and run tests have been available. As I wrote in Program Smarter, Not Harder, automated testing tools can provide the most bang for the buck in software development process improvement. After years of fighting software wars, developers have figured out that catching errors using static analysis relatively early in a development cycle rather than during user acceptance testing can mean the difference between project success and failure. These days, therefore, the biggest potential for process improvement may be in the concept of automating error prevention.

Static Analysis
Static analysis provides a code review by automatically inspecting source files to pinpoint code that could reduce application functionality, accessibility, and usability, or impair transformations and updates. As illustrated in Figure 1, clicking the Static Analysis button launches the analysis by applying the default or customized set of static analysis tools.

Static analysis uses tools that verify whether:

  • Files contain broken links and navigational problems
  • HTML, CSS, JavaScript, and ASP/VBScript code complies with W3C standards and industry guidelines
  • HTML and CSS code complies with Section 508 and W3C WAI accessibility guidelines
  • XML is well-formed and valid
  • HTML and XML files contain typos and misspelled words
In addition, static analysis can verify unique requirements that your team wants to check. For example, if your team needs to satisfy custom coding requirements, project-specific design requirements, and project-specific content requirements, you can automatically generate a rule that verifies whether each requirement is satisfied, and then check compliance during static analysis. As the code is analyzed, errors along with recommended fixes, are displayed.

You can customize static analysis tests by:

  • Enabling or disabling any of the built-in tools available for static analysis
  • Customizing the parameters of any tool applied during static analysis.
  • Designing tools that verify compliance with unique team or project requirements, then configuring WebKing to apply those tools during static analysis
  • Running static analysis on selected files or directories
  • Running static analysis on the click paths you record and generate for functional testing
  • Configuring WebKing to suppress error messages that are not relevant to your project
The single most powerful feature, however, is the ability to invoke WebKing's static analysis API from Java, JavaScript, or Python, as illustrated in Figure 1, thus making it possible to fully automate a code review. Whether you use WebKing or adopt another approach, whatever effort you put into creating an automated code review will be repaid manyfold in error prevention.

Functional Testing
Static analysis can make a big difference, but preventing bugs from showing up in deployed software requires automated functional testing. While static analysis finds problems by inspecting source code, functional testing finds problems by simulating how click paths would operate in a browser. Functional testing involves verifying whether designated click paths execute correctly. Functional testing exposes problems such as:

  • JavaScript runtime errors
  • Pop-up windows, page changes, and other effects that do not work as expected
  • Frames that do not load correctly in the context in which they are used
  • Valid pages that are blank or incomplete
  • Server-side program crashes and exceptions
  • Server errors and failures
  • Unexpected page content changes
  • Unexpected click path flow changes
WebKing facilitates functional testing by providing ways to generate and record test click paths without requiring any scripting. You can run functional tests by clicking the Test Functionality button and then use the Path Creation wizard to create the initial paths (see Figure 1). This wizard allows you to automatically generate paths or record specific paths by clicking through the application in a browser. WebKing then executes the initial paths and reports errors if it cannot execute one of the recorded path items, or if a runtime error occurs while executing the path. You can then extend this initial set of test paths in at least two ways:
  • Automatically record your application's most popular click paths by prompting WebKing to analyze your server log files.
  • Use the Path Creation wizard to automatically generate a variety of unique paths and/or to help you record specific click paths that correspond to your specification and use cases. By combining these options, you can quickly represent your most critical and popular paths as well as an array of other potential paths that might otherwise go untested.
Once you have verified that one or more paths work correctly, you can automatically record path execution results to establish baseline regression controls. After these regression controls are established, WebKing compares all subsequent path execution results to the baseline results, and then reports unexpected path flow and content changes. WebKing performs regression testing at the code level, so it exposes even subtle changes that are not detectable at the GUI level (for example, minor changes to scripts or hidden form options). It reports the differences both textually and graphically to help you pinpoint and understand any unexpected changes that occur. In addition, you can increase the scope and comprehensiveness of your functional testing by dynamically populating forms with values stored in a data source.

There are a number of ways to customize functional testing using WebKing, including populating forms with values stored in a data source, or recording and generating additional paths via the Path Creation wizard and GUI controls. Another useful feature is the ability to generate a site's most common paths by prompting WebKing to analyze its server log files. You can also extend the existing paths via JavaScript.

WebKing can be configured to run a script or tool every time it accesses a designated path step - for example, to perform initialization, set or remove a cookie, etc. It can also be set up to create regression controls for pages returned after forms are populated with data source values and then submitted.

After configuring WebKing to populate a path's forms with data source values, it will dynamically populate the designated forms every time it accesses that path. In addition, if you configure a test suite that exercises the path, WebKing will automatically populate and execute that path once for every available combination of data source rows. Consequently, if the path's forms use values from a single data source, that path is executed one time for every row in the data source. If the path's forms use values from multiple data sources, that path is executed and tested one time for every possible combination of rows. As a result, you can create and test a large number of path instances with relatively minimal effort.

Load Testing
Load testing involves exercising your application with virtual users and verifying whether it supports the tested traffic levels, patterns, and combinations. It is typically used to verify whether the application will perform adequately under the expected loads and surges, determine system capacity, and identify the source of any bottlenecks.

WebKing facilitates load testing by providing "intelligent" virtual users and sophisticated ready-to-run load test scenarios. To run a load test, you can click the Load Test button, and then choose one of the four preconfigured load test scenarios (bell curve, buffer test, steady load, or linear increase) that are included with WebKing. This initial test then simulates the selected load pattern and virtual users generate traffic by automatically generating paths through the application.

If you want to verify how your application performs under its typical load distributions and traffic combinations, you can prompt WebKing to automatically generate a custom load test based on your application's actual usage patterns.

Tell WebKing where to locate your server log files and it creates a test scenario that simulates your application's actual load distributions and your users' actual path choices. In addition, WebKing allows you to distribute virtual users across remote server machines to simulate extremely large loads and/or test from different locations.

By placing machines at strategic locations inside and outside the network, you can perform stress testing on specific pipes into the system and determine the capacity of each pipe, as well as the total system capacity. For example, imagine that you want to determine the source of a reported bottleneck.

First, you might configure several WebKing server machines immediately behind your Web server in order to determine the capacity of that link into your application. You would then use the WebKing client to design a test that ran a linear increase of virtual users on these machines.

By running the test and watching the result, you could determine if and when the increased number of virtual users caused performance degradation. You could then apply this same strategy to determine the capacity of each individual link to the application, and move progressively farther from the Web server until you were testing from remote locations far outside the network.

You can use the Confirm Scenario Wizard to select which scenario you want to use. Consider the following points when making the selection:

  • The Bell scenario simulates the typical daily user distribution (starts at the lowest point immediately after midnight, rises gradually in the morning, peaks towards the middle of the day, declines gradually in the afternoon, then returns to the lowest point at the end of the day). This is useful for expected usage testing, which determines how the site handles normal load patterns.
  • The Buffer Test scenario (illustrated in Figure 2) simulates a variable load over time. This helps determine whether resources are released when the load decreases. It also helps you determine whether overall performance degrades over time.
  • The Linear Increase scenario simulates a load that increases linearly over time. This is useful for stress capacity testing, which helps you determine how many users your application can support.
  • The Steady Load scenario simulates a steady load of users over time. This is useful for endurance testing, which helps you determine whether performance degrades over time.
WebKing will run a load test based on the specified scenario settings as well as the related virtual user profile settings and machine settings. Using default settings, WebKing will run the test on the local machine and create virtual users that create unique paths through the application.

Results are reported in a variety of customizable formats. When the test completes, WebKing will show a test summary in the right panel. To access results in bar graph, text format, or histogram format, choose the appropriate command from the Views box. These results can be sorted, filtered, and customized.

You can customize load tests by:

  • Prompting WebKing to automatically generate a custom load test scenario that simulates your application's actual load distributions and your users' actual path choices (based on log file analysis)
  • Customizing the paths virtual users follow during the test (virtual users can follow specific paths recorded or generated for functional testing, or they can automatically create original paths through the application)
  • Customizing the tools virtual users apply to verify functionality under load
  • Customizing virtual user browsing characteristics such as delay between clicks, types of links followed, cookie use, etc.
  • Configuring WebKing to perform stress testing by running tests from multiple server machines located at different locations within and without the network
  • Customizing the number and distribution of virtual users over time and over machines
  • Customizing the type and distribution of path combinations over time and over machines
  • Customizing the test duration
  • Designing a filter that focuses results on the details most important to you
  • Using Windows monitors and SNMP monitors to collect network information and system performance data during load testing
  • Using WebKing Windows services so you can monitor, start, stop, and synchronize load tests on remote server machines from a client machine - without having to manually start WebKing on each server machine
  • Using "Do not parse HTML" mode to generate a much higher load on your server with fewer machines running WebKing (when generating load is a higher priority than functional testing)
Parasoft's WebKing runs on Windows, Linux, or Solaris; it can be a very useful set of tools for preventing bugs from infiltrating software before deployment. It has an impressive static analysis capability, flexible functional testing features, and relatively easy-to-use load testing facilities.

More Stories By Jay Johnson

Jay Johnson is a J2EE architect at Qorusoft.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.