#ShadowClone #Unleash #Power #Cloud
ShadowClone is designed to delegate time consuming tasks to the cloud by distributing the input data to multiple serverless functions (AWS Lambda, Azure Functions etc.) and running the tasks in parallel resulting in huge performance boost!
ShadowClone uses IBM’s awesome Lithops library to distribute the workloads to serverless functions which is at the core of this tool. Effectively, it is a proof-of-concept script showcasing the power of cloud computing for performing our regular pentesting tasks.
- DNS Bruteforce using a very large wordlist within seconds
- Fuzz through a huge wordlist using ffuf on a single host
- Fuzz a list of URLs on a single path all from different IP addresses
- Port scan thousands of IPs in seconds
- Run a nuclei template on a list of hosts
- AWS/GCP/Azure/IBM cloud Account
- Docker installed on local machine. (required for initial setup only)
- Python 3.8+
Although the final script is cloud agnostic and should work with any supported platform, I have only tested it on AWS so far. Instructions for setting up GCP, Azure and IBM cloud environments will be added soon.
- Login to your AWS account and get API credentials (access key & secret)
- Go to IAM in AWS console and create a new policy with the following permissions:
- Create a new role with “Lambda” use case and attach the above policy to it.
- Keep a note of the ARN of this role, you will need it later.
- Go to S3 and create two buckets in the same region where your lambda is going to be executed.
- One bucket is used for storing logs, runtime information etc. and the other bucket will be used for storing uploaded files
If you are using AWS and would like to control the costs to remain in the free tier budget, I highly recommend following this article and setting up some budgets and alerts.
- Ensure docker is installed on your local machine. This is required for the initial setup only.
- Clone the repo and install python dependencies
git clone https://github.com/fyoorer/ShadowClone.git
python -m venv env
pip install -r requirements.txt
All the magic happens in the lithops library, which should be installed after the previous command.
- Verify that
lithopscommand line utility is installed by running
⚡ lithops test
2022-01-18 08:08:45,832 [INFO] lithops.config — Lithops v2.5.8
2022-01-18 08:08:45,833 [INFO] lithops.storage.backends.localhost.localhost — Localhost storage client created
2022-01-18 08:08:45,833 [INFO] lithops.localhost.localhost — Localhost compute client created
2022-01-18 08:08:45,833 [INFO] lithops.invokers — ExecutorID b9419a-0 | JobID A000 – Selected Runtime: python
2022-01-18 08:08:45,833 [INFO] lithops.invokers — Runtime python is not yet installed
2022-01-18 08:08:45,833 [INFO] lithops.localhost.localhost — Extracting preinstalled Python modules from python
2022-01-18 08:08:46,110 [INFO] lithops.invokers — ExecutorID b9419a-0 | JobID A000 – Starting function invocation: hello() – Total: 1 activations
2022-01-18 08:08:46,111 [INFO] lithops.invokers — ExecutorID b9419a-0 | JobID A000 – View execution logs at /tmp/lithops/logs/b9419a-0-A000.log
2022-01-18 08:08:46,111 [INFO] lithops.wait — ExecutorID b9419a-0 – Getting results from functions
2022-01-18 08:08:48,125 [INFO] lithops.executors — ExecutorID b9419a-0 – Cleaning temporary data
Hello fyoorer! Lithops is working as expected 🙂
If you see this, that means Lithops is installed and working as intended.
- Now to make lithops work with your cloud provider, create a configuration file at
~/.lithops/configand copy the following content into it
access_key_id: AKIA[REDACTED] #changeme
secret_access_key: xxxx[REDACTED]xxxx #changeme
#account_id: # Optional
execution_role: arn:aws:iam::123123123123:role/lithops-execution-role #changeme
storage_bucket: mybucket #changeme
region_name : us-east-1
The lines marked with
#changeme need to be updated with the values noted above
secret_access_key– Your account’s API credentials
execution_role– Enter the IAM Role ARN noted above
storage_bucket– Enter the name of the bucket you wish to use for storing logs
Ensure that the config file is placed at
Build the image using
lithops build command:
lithops runtime build sc-runtime -f Dockerfile
Next, register the runtime in your cloud environment with the following command:
lithops runtime create sc-runtime –memory 512 –timeout 300
Check runtime successfully registered
lithops runtime list
Copy the runtime name displayed in the output. We will need it in the next step.
Finally, update the
config.py with the name of your runtime and the bucket:
LITHOPS_RUNTIME=”lithops_v2-5-8_ke73/sc-runtime” #runtime name obtained from above
STORAGE_BUCKET=”mytestbucket” #name of the 2nd bucket created above
Finally we are ready run some lambdas!
python shadowclone.py -h
usage: cloudcli.py [-h] -i INPUT [-s SPLITNUM] [-o OUTPUT] -c COMMAND
-h, –help show this help message and exit
-i INPUT, –input INPUT
-s SPLITNUM, –split SPLITNUM
Number of lines per chunk of file
-o OUTPUT, –output OUTPUT
-c COMMAND, –command COMMAND
command to execute
How it works
We create a container image during the initial setup and register it as a runtime for our function in AWS/GCP/Azure whatever. When you execute ShadowClone on your computer, instances of that container are activated automatically and are only active for the duration of its execution. How many instances to activate is dynamically decided at runtime depending on the size of the input file provided and the split factor. The input is then split into chunks and equally distributed between all the instances to execute in parallel. For example, if your input file has 10,000 lines and you set the split factor to 100 lines, then it will be split into 100 chunks of 100 lines each and 100 instances will be run in parallel!
- Extremely fast
- No need to maintain a VPS (or a fleet of it :))
- Costs almost nothing per month
- Compatible with free tiers of most cloud services
- Cloud agnostic
- Same script works with AWS, GCP, Azure etc.
- Supports upto 1000 parallel invocations
- Dynamically decide the number of invocations
- Run any tool in parallel on the cloud
- Pipe output to other tools
This tool was inspired by the awesome Axiom and Fleex projects and goes beyond the concept of VPS for running the tools by using serverless functions and containers.
|Cost||Per instance/per minute||Mostly Free**|
|Startup Time||4-5 minutes||2-3 seconds|
|Max Execution Time||Unlimited||15 minutes|
|On Demand Scalability||No||∞|
*Most cloud providers do not allow spinning up too many instances by default, so you are limited to around 10-15 instances at max. You have to make a request to the support to increase this number.
** AWS & Azure allow 1 million invocations per month for free. Google allows 2 million invocations per month for free. You will be charged only if you go above these limits