URL Rewrite Rules
Web application developers employ URL Rewrite Rules to hide parameters within the URL path structure. This practice facilitates comprehensive indexing by search engines while presenting URLs to web browsers in a user-friendly format. For example, when navigating an online hardware store, the URL typically appears as http://www.example.com/tools/hammer/.
Through a URL rewrite rule, the web server transforms this URL into a specific format, such as http://www.example.com/library.php?tools=hammer. This enables the retrieval of data from the backend database to display tool details to visitors.
In this scenario, the subdirectory ("/tools") in the initial URL functions as a parameter within the library.php file, accommodating inputs like the tool name ("hammer"). Acunetix 360 conducts scans by sending standard HTTP requests to simulate attacker behavior, ensuring the web application accepts such requests and appropriately scans all parameters within the URLs. Furthermore, it can scan pages with multiple parameters in the URL.
NOTE Acunetix 360 automatically detects URL rewrites on the target website using heuristic methods. Additionally, it offers automatic configuration of settings. Nevertheless, manual configuration of URL Rewrite Rules, as detailed in this document, can enhance the efficiency of the scan. |
How to Configure URL Rewrite Rules in Acunetix 360
- Select Scans > New Scan from the left-side menu.
- Specify the Target URL and Scan Profile.
- In the Scan Settings section, select URL Rewrite.
- In the URL Rewrite Mode, select from the options: None, Heuristic, or Custom.
TIP: Refer to the URL Rewrite Fields information at the end of this document for more information about these settings. |
- Heuristic is the default mode and automatically populates these fields:
- Root Path Maximum Dynamic Signatures
- Sub Path Maximum Dynamic Signatures
- Block Separators
- Analyzable Extensions
- None applies no rules.
- Custom requires the following configurations:
- Select the checkbox to Enable Heuristic URL Rewrite Detection to automatically determine additional URL rewrite rules. When enabled, both custom and heuristic rules will apply. If disabled, only the custom rules will apply.
- Click + New to add a new Placeholder Pattern and RegEx Pattern.
- Optionally, click + New in the Exclusions section to enter an Excluded Path and mark it as Is Regex if applicable.
- Continue other Scan Settings as required, then click Launch to start a scan with these settings. Alternatively, click Save Profile to save these settings for use on a future scan.
Challenges Associated with URL Rewrite Rules
This table outlines and elucidates the potential issues encountered by automated web vulnerability scanners when scanning websites utilizing URL Rewrite Rules.
Issue | Challenge | Context |
Parameters within URLs are overlooked during scanning due to misidentification | Web scanners struggle with URL rewriting, mistaking parameters for directories and leaving them unscanned. | For example, the URL http://www.example.com/tools/hammer/ is misinterpreted, as "tools" and "hammer" are considered directories instead of parameters and values, respectively. |
Extended scan | Extended scans can lead to inaccurate results and software crashes. For instance, if a web vulnerability scanner fails to recognize parameters and values in URLs, it may treat each item in a tool database as a separate page to crawl and scan. Inadequate handling of memory problems and exceptions may further contribute to crashes, resulting in lost results and wasted time. | Inadequate handling of memory problems and other exceptions in your scanner may lead to crashes, resulting in lost results and wasted time. Failure to configure URL rewrite rules in Acunetix 360 leads to heuristic pattern identification, limiting scans to prevent prolonged durations and inaccurate outcomes. |
Setting up URL rewrite rules presents a challenging task | Commercial web vulnerability scanners often offer configuration options to identify parameters within URLs due to the prevalence of URL rewrite technology in web applications. However, users face challenges such as complex setup processes, the need for knowledge in writing regular expressions, and requiring access to web server configuration files. | Configuring URL rewrite rules is particularly challenging for users without deep understanding of the web application or direct access to configuration files, making it a time-consuming task even for those with expertise. |
Web applications are not properly scanned for vulnerabilities | After configuring URL rewrite rules in your web vulnerability scanner, additional limitations emerge in scanning the web application. Web applications, as a security measure, reject HTTP requests that are already 'translated', like http://www.example.com/library.php?tools=hammer. This is default behavior for .NET web applications, which worsens the issue when scanning MVC web applications due to their distinct URL rewriting approach. | Acunetix 360 scans MVC web applications, but numerous other vulnerability scanners fail to do so, even with configured URL rewrite rules. After setting up URL rewrite rules in your scanner, it sends translated query HTTP requests. Despite the security scanner reporting a successful scan, most HTTP requests are denied, leaving parameters in URLs unscanned and creating a misleading sense of security. |
URL Rewrite Fields
This table lists and describes the fields in the URL Rewrite tab.
Field | Description |
Root Path Max Dynamic Signatures | If a URL block in the root path contains more items than this limit, it will be identified as a URL rewrite parameter. It must be between 1 and 10,000. This field is displayed only in the Heuristic tab. Example:
If Root Path Max Dynamic Signatures is set to 3, and you have more than 3 unique categories (electronics, clothing, home-garden, books, toys, etc.), Invicti will treat the category as a dynamic parameter. |
Sub Path Dynamic Signatures | If a URL block in the subpath contains more items than this limit, it will be identified as a URL rewrite parameter. It must be between 1 and 10,000. This field is displayed only in the Heuristic tab. Example:
If Sub Path Dynamic Signatures is set to 50, and you have more than 50 unique authors or book titles, these will be treated as dynamic parameters. |
Block Separators | Enter separators to use to split the URL into blocks. This field is displayed only in the Heuristic tab. Example:
If Block Separators is set to "-_", Invicti will split the URL into blocks: [2023, 07, 15] [technology] [new, smartphone, release]. |
Analyzable Extensions | If the URL contains a file extension, it will be analyzed only if the respective extension is in this list. This field is displayed only in the Heuristic tab. |
Enable Heuristic URL Rewrite detection | Acunetix 360 will try to automatically detect other URL rewrite rules if this option is set. This field is displayed only in the Custom tab. |
Placeholder Pattern | This contains the relative path with placeholders for URL rewrite parameters. This field is displayed only in the Custom tab. |
RegEx Pattern | This is a regular expression used for matching the URL rewrite parameters. This field is displayed only in the Custom tab. |