-
Notifications
You must be signed in to change notification settings - Fork 80
1.4 Filtering Rules
ThreatPinch comes with a number of filtering rule capabilities to help you control how and where your API requests are used. This section will explain the different options and then provide a few real world examples of how to use the filtering rules.
ThreatPinch has a number of filters that are designed to either allow or deny at the global plugin level, or at the Request Group level. Allow filters are ALWAYS taken into consideration first. If you set your global settings allow list to only include *.google.com and any of your request groups are set to allow mysiem.fqdn your request groups will never trigger. If this is confusing, skip to some of the examples at the bottom of this section.
The ThreatPinch global settings schema has two array fields that are regex enabled match fields. These fields allow you to control either only the current page URLs that Threatpinch is allowed or denied from running on. If you add even one URL to the allow list, ThreatPinch will only work on that one URL. Likewise if there are any URLs in the deny list, ThreatPinch will not progress past the point of determine the URL of the current page.
Request Groups give you the power to ensure API requests are only performed on the desired resources. Meaning that Request Lookups belong to Request Groups and will only be triggered if the fit within the predefined filtering rules you lay out.
Straight from the Webstore ThreatPinch comes with two pre-configured Request Groups, INTERNET and LOCALNET. All the pre-configured Request Lookups that come with ThreatPinch are part of the INTERNET Request Group. The LOCALNET Request Group is for you to either add your internal asset database lookups, or any other locally available resources you wish to add. More on this further down.
Request Group allow/deny lists work similarly to the global lists, however they are only enforced for Request Lookups that belong to that particular Request Group.
Request Groups also have the ability to either allow or deny a range of IP addresses configured by their CIDR notation, or if you are only interested in having a request group work on RFC1918 ranges, you can set the rfc1918 flag to either be enabled or disabled.
The most common use for the CIDR allow/deny lists is to put your external IP address range in the INTERNET Request Group under the deny list. This will prevent ThreatPinch from accidentally looking up a (hopefully) known good IP address in various threat feeds. This is good for two reasons:
- API Requests are not wasted on IP addresses that you should likely already have internal asset information for. If you know your external ranges, you should ensure that you perform lookups for those ranges in your LOCALNET request group.
- Threat feeds likely do analytics on searches, and you don't want to be flooding their search API with your external IP addresses. They might start wondering why so many searches are performed for those ranges and think something is wrong.
For Example lets say I worked at GitHub and wanted to prevent my external IP ranges from being looked up in the preconfigured INTERNET Request Group which contains Request Lookups in free threat feeds. I would set my cidrDenyList for the INTERNET Request Group to the following:
['192.30.252.0/22','207.97.227.239/32','204.232.175.90/32']Not yet implemented as we are still testing FQDN Request Lookups and regexes. If you want to help us test the FQDN features send me a tweet @threatpinch and I can send you the configurations. This functionality will work similar to the other filtering rules, however it will be focused on FQDN suffixes.
Below I will describe a few examples of how to get the most out of your Request Group filters.
For this configuration I will expand upon the GitHub employee example above. Lets assume the following requirements:
- Prevent my external IP ranges 192.30.252.0/22, 207.97.227.239/32, 204.232.175.90/32 from being looked up in external threat feeds.
- Perform internal asset lookups for my internal IP address range that consists only of the 10.0.0.0/8 CIDR range.
- Only perform asset lookups on webpages that have the internal.github.com domain suffix.
Request Group configurations will look like this:
[
{
"cidrAllowList": ["10.0.0.0/8"],
"cidrDenyList": [],
"fqdnAllowList": [],
"fqdnBlacklist": [],
"name": "LOCALNET",
"rfc1918": true,
"websiteAllowList": [".*internal.github.com"],
"websiteDenyList": []
},
{
"cidrAllowList": [],
"cidrDenyList": [
"192.30.252.0/22",
"207.97.227.239/32",
"204.232.175.90/32"
],
"fqdnAllowList": [],
"fqdnBlacklist": [],
"name": "INTERNET",
"rfc1918": false,
"websiteAllowList": [],
"websiteDenyList": []
}
]Continuing to expand on the example, lets pretend that GitHub has a more complex environment and they have multiple asset databases for DEV/UAT/PROD.
- Prevent my external IP ranges 192.30.252.0/22, 207.97.227.239/32, 204.232.175.90/32 from being looked up in external threat feeds.
- Perform internal asset lookups for my internal IP address range that consists only of the 10.0.0.0/8 CIDR range.
- DEV assets are located on the 10.0.1.0/24 network range.
- UAT assets are located on the 10.0.2.0/24 network range.
- PROD assets are located on the 10.0.3.0/24 network range.
- Only perform asset lookups for DEV assets on webpages that have the dev.github.com domain suffix.
- Only perform asset lookups for UAT assets on webpages that have the uat.github.com domain suffix.
- Only perform asset lookups for PROD assets on webpages that have the prod.github.com domain suffix.
- Perform asset lookups for external IP addresses in PROD asset db.
Request Group configurations might look something like this:
[
{
"cidrAllowList": ["10.0.1.0/24"],
"cidrDenyList": [],
"fqdnAllowList": [],
"fqdnBlacklist": [],
"name": "DEV",
"rfc1918": true,
"websiteAllowList": [".*dev.github.com"],
"websiteDenyList": []
},
{
"cidrAllowList": ["10.0.2.0/24"],
"cidrDenyList": [],
"fqdnAllowList": [],
"fqdnBlacklist": [],
"name": "UAT",
"rfc1918": true,
"websiteAllowList": [".*uat.github.com"],
"websiteDenyList": []
},
{
"cidrAllowList": [
"10.0.3.0/24",
"192.30.252.0/22",
"207.97.227.239/32",
"204.232.175.90/32"
],
"cidrDenyList": [],
"fqdnAllowList": [],
"fqdnBlacklist": [],
"name": "PROD",
"rfc1918": true,
"websiteAllowList": [".*prod.github.com"],
"websiteDenyList": []
},
{
"cidrAllowList": [],
"cidrDenyList": [
"192.30.252.0/22",
"207.97.227.239/32",
"204.232.175.90/32"
],
"fqdnAllowList": [],
"fqdnBlacklist": [],
"name": "INTERNET",
"rfc1918": false,
"websiteAllowList": [],
"websiteDenyList": []
}
]