Skip to content

Commit

Permalink
some small fixes
Browse files Browse the repository at this point in the history
  • Loading branch information
brifordwylie committed Oct 10, 2018
1 parent 892fc41 commit d41f852
Show file tree
Hide file tree
Showing 3 changed files with 85 additions and 84 deletions.
6 changes: 6 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Set the default behavior, in case people don't have core.autocrlf set.
* text=auto

# Denote all files that are truly binary and should not be modified.
*.png binary
*.jpg binary
43 changes: 19 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,15 @@
Bro Analysis Tools (BAT) [![travis](https://travis-ci.org/SuperCowPowers/bat.svg?branch=master)](https://travis-ci.org/SuperCowPowers/bat) [![Coverage Status](https://coveralls.io/repos/github/SuperCowPowers/bat/badge.svg?branch=master)](https://coveralls.io/github/SuperCowPowers/bat?branch=master) [![supported-versions](https://img.shields.io/pypi/pyversions/bat.svg)](https://pypi.python.org/pypi/bat) [![license](https://img.shields.io/badge/License-Apache%202.0-green.svg)](https://choosealicense.com/licenses/apache-2.0)
===============================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================
## Bro Analysis Tools (BAT) [![travis](https://travis-ci.org/SuperCowPowers/bat.svg?branch=master)](https://travis-ci.org/SuperCowPowers/bat) [![Coverage Status](https://coveralls.io/repos/github/SuperCowPowers/bat/badge.svg?branch=master)](https://coveralls.io/github/SuperCowPowers/bat?branch=master) [![supported-versions](https://img.shields.io/pypi/pyversions/bat.svg)](https://pypi.python.org/pypi/bat) [![license](https://img.shields.io/badge/License-Apache%202.0-green.svg)](https://choosealicense.com/licenses/apache-2.0)


The BAT Python package supports the processing and analysis of Bro data
with Pandas, scikit-learn, and Spark

BroCon 2017 Presentation
------------------------
## BroCon 2017 Presentation

Data Analysis, Machine Learning, Bro, and You!
([Video](https://www.youtube.com/watch?v=pG5lU9CLnIU))

Why BAT?
--------
## Why BAT?

Bro already has a flexible, powerful scripting language why should I use
BAT?
Expand All @@ -25,10 +23,9 @@ bridge from raw Bro data to packages like Pandas, scikit-learn, and
Spark. We also have example notebooks that show step-by-step how to get
from here to there.

Example: Pull in Bro Logs as Python Dictionaries
------------------------------------------------
###Example: Pull in Bro Logs as Python Dictionaries

``` {.sourceCode .python}
```python
from bat import bro_log_reader
...
# Run the bro reader on a given log file
Expand All @@ -52,10 +49,9 @@ types properly converted.
'uid': 'CJsdG95nCNF1RXuN5'}
...

Example: Bro log to Pandas DataFrame (in one line of code)
----------------------------------------------------------
### Example: Bro log to Pandas DataFrame (in one line of code)

``` {.sourceCode .python}
```python
from bat.log_to_dataframe import LogToDataFrame
...
# Create a Pandas dataframe from a Bro log
Expand All @@ -68,17 +64,17 @@ from bat.log_to_dataframe import LogToDataFrame
**Output:** All the Bro log data is in a Pandas DataFrame with proper
types and timestamp as the index

query id.orig_h id.orig_p id.resp_h \

> ts 2013-09-15 17:44:27.631940 guyspy.com 192.168.33.10 1030 4.2.2.3
> 2013-09-15 17:44:27.696869 www.guyspy.com 192.168.33.10 1030 4.2.2.3
> 2013-09-15 17:44:28.060639 devrubn8mli40.cloudfront.net 192.168.33.10
> 1030 4.2.2.3 2013-09-15 17:44:28.141795 d31qbv1cthcecs.cloudfront.net
> 192.168.33.10 1030 4.2.2.3 2013-09-15 17:44:28.422704 crl.entrust.net
> 192.168.33.10 1030 4.2.2.3
```
query id.orig_h id.orig_p id.resp_h
ts
2013-09-15 17:44:27.631940 guyspy.com 192.168.33.10 1030 4.2.2.3
2013-09-15 17:44:27.696869 www.guyspy.com 192.168.33.10 1030 4.2.2.3
2013-09-15 17:44:28.060639 devrubn8mli40.cloudfront.net 192.168.33.10 1030 4.2.2.3
2013-09-15 17:44:28.141795 d31qbv1cthcecs.cloudfront.net 192.168.33.10 1030 4.2.2.3
2013-09-15 17:44:28.422704 crl.entrust.net 192.168.33.10 1030 4.2.2.3
```

More Examples
-------------
## More Examples

- Easy ingestion of any Bro Log into Python (dynamic tailing and log
rotations are handled)
Expand All @@ -92,8 +88,7 @@ More Examples
Examples](https://bat-tools.readthedocs.io/en/latest/examples.html)
for more details.

Analysis Notebooks
------------------
## Analysis Notebooks

BAT enables the processing, analysis, and machine learning of realtime
data coming from Bro.
Expand Down
120 changes: 60 additions & 60 deletions setup.py
Original file line number Diff line number Diff line change
@@ -1,60 +1,60 @@
#!/usr/bin/env python
"""Setup.py for bat"""

import os

from setuptools import setup, find_packages

readme = open('README.rst').read()

# Pull in the package info
package_name = 'bat'
package = __import__(package_name)
version = package.__version__
author = package.__author__
email = package.__email__


# Data and Example Files
def get_files(dir_name):
"""Simple directory walker"""
return [(os.path.join('.', d), [os.path.join(d, f) for f in files]) for d, _, files in os.walk(dir_name)]


setup(
name=package_name,
version=version,
description='Bro Analysis Tools',
long_description=readme,
author=author,
author_email=email,
url='https://github.com/kitware/bat',
packages=find_packages(),
include_package_data=True,
data_files=get_files('data') + get_files('examples'),
install_requires=[
'requests',
'watchdog',
'numpy',
'scipy',
'pandas',
'scikit-learn',
'pyarrow'
],
extras_require={
'all': ['yara-python', 'tldextract']
},
license='Apache',
keywords='Bro, Python, Networking, Security, Scikit-Learn, Spark, Kafka, Parquet',
classifiers=[
'Development Status :: 2 - Pre-Alpha',
'Intended Audience :: Developers',
'License :: OSI Approved :: Apache Software License',
'Natural Language :: English',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: Implementation :: CPython',
'Programming Language :: Python :: Implementation :: PyPy'
],
)
#!/usr/bin/env python
"""Setup.py for bat"""

import os

from setuptools import setup, find_packages

readme = open('README.md').read()

# Pull in the package info
package_name = 'bat'
package = __import__(package_name)
version = package.__version__
author = package.__author__
email = package.__email__


# Data and Example Files
def get_files(dir_name):
"""Simple directory walker"""
return [(os.path.join('.', d), [os.path.join(d, f) for f in files]) for d, _, files in os.walk(dir_name)]


setup(
name=package_name,
version=version,
description='Bro Analysis Tools',
long_description=readme,
author=author,
author_email=email,
url='https://github.com/SuperCowPowers/bat',
packages=find_packages(),
include_package_data=True,
data_files=get_files('data') + get_files('examples'),
install_requires=[
'requests',
'watchdog',
'numpy',
'scipy',
'pandas',
'scikit-learn',
'pyarrow'
],
extras_require={
'all': ['yara-python', 'tldextract']
},
license='Apache',
keywords='Bro, Python, Networking, Security, Scikit-Learn, Spark, Kafka, Parquet',
classifiers=[
'Development Status :: 2 - Pre-Alpha',
'Intended Audience :: Developers',
'License :: OSI Approved :: Apache Software License',
'Natural Language :: English',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: Implementation :: CPython',
'Programming Language :: Python :: Implementation :: PyPy'
],
)

0 comments on commit d41f852

Please sign in to comment.