Opinion: DevOps are simply afraid of trying something new. They are used to Selenium tests that hog the pipelines and provide hard-to-interpret results but at the same time they often shun DAST testing, which is nowhere near as troublesome.
Recently, I had an interesting discussion with a good friend of mine, Michael M., who happens to be a senior QA tester at a mid-sized company that develops advanced B2B supply chain planning web applications. I simply wanted to learn more about their QA testing process. I felt that they must have it handled pretty well because this company is renowned for hiring the best graduates from the best universities in the country. It is known to be innovative, agile, and otherwise a good example of how things should be done in the industry.
The longer I was talking to Michael, the more shocked I was to learn that the QA process is not as easy and as smooth as I thought it would be.
Since their applications are fully developed in Java, their QA processes involve three types of tests. First, there are simple unit tests that are written in Java itself. Basically, a test accesses a single function, feeds it with example data, takes the output values, and compares the output values to expected values. Nothing more.
Then, there are functional and integration Selenium tests. These tests go through all screens and basically attempt to make Selenium act like a simple crawler. That crawler, by design, does not reach into administration, configuration, or reporting screens at all.
The DOM structure resulting from each Selenium action is compared with expected data (recorded manually earlier). If there is a discrepancy, Selenium takes a “screenshot” of the application and basically tells the developer “there’s an error somewhere here on this screen, find it and fix it”.
The whole QA testing process for their primary product takes… approximately 4 hours. Most of this time is consumed by the Selenium tests. The process would take at least twice as much if they didn’t cut down on creating new Docker containers. After each test, they simply run a series of SQL commands and load up the default database content instead of loading up the whole environment.
Before talking to Michael, my knowledge of QA testing was rather theoretical. After the talk, I felt shocked. I never expected that Selenium tests would take so long and that the poor developer would only have a screenshot as a guide!
And then, two thoughts came to my mind.
The first thought was actually a question. Why are DevOps managers wary of DAST testing in the SDLC? Why do they say that it takes a long time when their Selenium tests take much longer? Why do they complain that DAST does not point to a line of code while their Selenium tests just give developers a screenshot of the application?
I can only guess that it’s simply because DAST testing in SDLC is new to them or they had some bad experiences with trying to run free DAST products. From my point of view, knowing how Acunetix works, a DAST test in the SDLC would take a fraction of the time that is eaten up by Selenium and would provide the developers with much more information about the error! That would, of course, not fix problems with Selenium but it would not hog the pipelines any more than they are already.
The second thought that came to my mind was associated with some marketing pitches that I read, coming from a renowned passive IAST tool manufacturer. Those marketing materials were falsely promoting their product as more thorough than an active/true IAST. If Michael’s company’s tests completely skip all administrative, configuration, and reporting screens, how secure would their application be if it was tested using passive IAST? After all, security testing would then be completely ignoring a major part of the functionality of the product.
After the talk, I came to two conclusions. First of all, it’s not the capabilities of DAST products that are the true reason why many businesses fail to implement early testing. Second of all, I confirmed my belief that passive IAST is not a good way to keep your application secure.
Get the latest content on web security
in your inbox each week.
THE AUTHOR
Principal Cybersecurity Writer
