Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to Version 1.4 #5

Closed
wants to merge 36 commits into from
Closed
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
1a5bb1b
Pretty Major Refactor to use the version 1.4 of the Adobe Analytics API.
dancingcactus Jun 15, 2014
6291ff9
Create LICENSE.md
dancingcactus Jun 15, 2014
3ec405b
Create LICENSE.md
dancingcactus Jun 15, 2014
45148b7
Clean up and docs
dancingcactus Jun 16, 2014
091259d
Clean up on the projects
dancingcactus Jun 16, 2014
86b82b7
Fix bugs in the unittests
dancingcactus Jun 16, 2014
98eb2ce
Small update to logging
dancingcactus Jun 16, 2014
ec6f8f1
Merge branch 'api1_4' of https://github.com/dancingcactus/python-omni…
dancingcactus Jun 16, 2014
ed3160d
Merge pull request #1 from dancingcactus/api1_4
dancingcactus Jun 16, 2014
be03c1b
Updated install instructions
dancingcactus Jun 16, 2014
4087b9b
Merge pull request #2 from dancingcactus/api1_4
dancingcactus Jun 16, 2014
4c4e3a5
Added support for segmentation (through ID) and cleaned up the loggin…
dancingcactus Jun 17, 2014
0167635
Merge pull request #3 from dancingcactus/api1_4
dancingcactus Jun 17, 2014
58ac12d
Fix issue #7, adding weekly granularity
dancingcactus Jun 20, 2014
9070ea4
Version bump
dancingcactus Jun 20, 2014
207adfc
Merge pull request #8 from dancingcactus/fix_granularity
dancingcactus Jun 20, 2014
884aab0
Rough support for inline segments
dancingcactus Jun 21, 2014
1028402
Merging in master branch
dancingcactus Jun 21, 2014
c20768c
Merge branch 'inline-segments'
dancingcactus Jun 21, 2014
8c17f53
Added more doc strings to help when inspecting objects in ipython usi…
dancingcactus Jun 21, 2014
43ece1d
Update README.md with more information about how to use the library
dancingcactus Jun 23, 2014
b538abb
Get ready for the pull request
dancingcactus Jun 23, 2014
7be37f8
Parse the dates out into a datetime object. Especially useful when de…
dancingcactus Jun 23, 2014
342338f
Add pretty printing for iPython notebooks
dancingcactus Jun 27, 2014
9ed9d08
Added shortcuts for adding multiple metrics and elements
dancingcactus Jun 27, 2014
34512bf
Fix an issue with passing in multiple elements
dancingcactus Jul 8, 2014
87641c7
Fixed bug where the raw JSON of a response wasn't being completely pr…
dancingcactus Jul 10, 2014
e3416c0
fixed multi-byte character bugs
dancingcactus Jul 16, 2014
2e16416
Additional multibyte character fix
dancingcactus Jul 23, 2014
9fd40cf
Fix granularity in range as well as a few other fixes
dancingcactus Jul 24, 2014
2137892
Merge pull request #15 from dancingcactus/granularity-fix
dancingcactus Jul 24, 2014
fd81afd
adjusted logging and fixed a few bugs
dancingcactus Aug 22, 2014
403683c
version bump
dancingcactus Aug 22, 2014
0a4800b
Added a default heartbeat for the run method
dancingcactus Aug 22, 2014
827595d
Removed ipython notebook
dancingcactus Aug 22, 2014
9cadeb0
Merge pull request #16 from dancingcactus/master
dancingcactus Aug 22, 2014
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,5 @@
*.pyc
*.egg-info
dist
*-jgrover*
*.log
21 changes: 21 additions & 0 deletions LICENSE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
The MIT License (MIT)

Copyright (c) {{{year}}} {{{fullname}}}

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
177 changes: 137 additions & 40 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,97 +1,166 @@
# python-omniture

`python-omniture` is a wrapper around the Adobe Omniture web analytics API.
`python-omniture` is a wrapper around the Adobe Analytics API.

It is not meant to be comprehensive. Instead, it provides a high-level interface
to certain common kinds of queries, and allows you to do construct other queries
to certain many of the common reporting queries, and allows you to do construct other queries
closer to the metal.

## Installation

Through PyPI:
Through PyPI (older version):

pip install omniture

Latest and greatest:

pip install git+git://github.com/stdbrouw/python-omniture.git
pip install git+http://github.com/stdbrouw/python-omniture.git

## Authentication

The most straightforward way to authenticate is with:

```python
import omniture
account = omniture.authenticate('my_username', 'my_secret')
analytics = omniture.authenticate('my_username', 'my_secret')
```

However, to avoid hardcoding passwords, instead you can also put your username
and password in unix environment variables (e.g. in your `.bashrc`):

```bash
export OMNITURE_USERNAME=my_username
export OMNITURE_SECRET=my_secret
```

With your credentials in the environment, you can then log in as follows:

```python
import os
import omniture
account = omniture.authenticate(os.environ)
analytics = omniture.authenticate(os.environ)
```

## Account and suites

You can very easily access some basic information about your account and your
reporting suites:

```python
print analytics.suites
suite = analytics.suites['guardiangu-network']
suite = analytics.suites['reportsuite_name']
print suite
print len(suite.evars)
print suite.segments
print suite.metrics
print suite.elements
print suite.segments
```

You can refer to suites, segments, elements and so on using both their
human-readable name or their id. So for example `suite.segments['pageviews']` and `suite.segments['Page Views']` will work exactly the same. This is especially useful in cases when segment or metric identifiers are long strings of gibberish. That way you don't have to riddle your code with references to `evar16` or `custom4` and instead can call them by their title.
human-readable name or their id. So for example `suite.metrics['pageviews']` and `suite.metrics['Page Views']` will work exactly the same. This is especially useful in cases when segment or metric identifiers are long strings of gibberish. That way you don't have to riddle your code with references to `evar16` or `event4` and instead can call them by their title. Just remember that if you change the friendly name in the interface it will break your script.

## Running a report

`python-omniture` can run ranked, trended and "over time" reports
`python-omniture` can run ranked, trended and over time reports. Pathing reports are still in the works

Here's a quick example:

report = network.report \
.over_time(metrics=['pageviews', 'visitors']) \
.range('2013-05-01', '2013-05-31', granularity='month') \
.sync()
```python
report = suite.report \
.element('page') \
.metric('pageviews') \
.run()
```
This will generate the report definition and run the report. You can alternatively generate a report definition and save the report defintion to a variable by omitting the call to the `run()` method

If you call `print` on the report defintion it will print out the JSON that you can use in the [API explorer](https://marketing.adobe.com/developer/en_US/get-started/api-explorer) for debugging or to use in other scripts

```python
report = suite.report \
.element('page') \
.metric('pageviews') \

print report
```

### Report Options
Here are the options you can add to a report.

**element()** - `element('element_id or element_name', **kwargs)` Adds an element to the report. If you need to pass in additional information in to the element (e.g. `top` or `startingWith` or `classification`) you can use the kwargs to do so. A full list of options available is documented [here](https://marketing.adobe.com/developer/en_US/documentation/analytics-reporting-1-4/r-reportdescriptionelement). If multiple elements are present then they are broken-down by one another

**breakdown()** - `breakdown('element_id or element_name', **kwargs)` Same as element. It is included to make report queries more readable when there are multiple element. Use when there are more than one element. eg.

```python
report = suite.report.element('evar1').breakdown('evar2')
```

**metric()** - `metric('metric')` Adds a metric to the report. Can be called multiple times to add multiple metrics if needed.

**range()** - `range('start', 'end=None', 'months=0', 'days=0', 'granularity=None')` Sets the date range for the report. All dates shoudl be listed in ISO-8601 (e.g. 'YYYY-MM-DD')

* **Start** -- Start date for the report. If no stop date is specified then the report will be for a single day
* **Stop** -- End date for the report.
* **months** -- Number of months back to run the report
* **days** -- Number of days back from now to run the report
* **granularity** -- The Granularity of the report (`hour`, `day`, `week`, `month`)

**granularity()** -- `granularity('granularity')` Set the granularity of the report

**sortBy()** -- `sortBy('metric')` Set the sortBy metric

**filter()** -- `filter('segment')` or `filter(element='element', selected=[])` Set the segment to be applied to the report. Can either be an segment id/name or can be used to define an inline segment by specifying the paramtered. You can add multiple filters if needed and they will be stacked (anded together)
```python
report = suite.report.filter('537d509ee4b0893ab30616c7')
report = suite.report.filter(element='page', selected=['homepage'])
report = suite.report.filter('537d509ee4b0893ab30616c7')\
.filter(element='page', selected=['homepage'])
```

**currentData()** --`currentData()` Set the currentData flag

Some basic features of the three kinds of reports you can run:
**run()** -- `run()` Run the report and check the queue until done

* over_time
* supports multiple metrics but only one element: time
* useful if you need information on a per-page basis
* supports hourly reporting (and up)
* ranked
* ranks pages in relation to the metric
* one number (per metric) for the entire reporting period
* only supports daily, weekly and monthly reporting
* trended
* movement of a single element and metric over time (e.g. visits to world news over time)
* supports hourly reporting (and up)
**set()** -- `set(key, value)` Set a custom attribute in the report definition

Accessing the data in a report works as follows:
## Using the Results of a report

report.data['pageviews']
To see the raw output of a report.
```python
print report
```

If you need an easy way to access the data in a report:

```python
data = report.data
```

This will generate a list of dicts with the metrics and elements called out by id.


### Pandas Support
`python-omniture` can also generate a data frame of the data returned. It works as follows:

```python
df = report.dataframe
```

Pandas Data frames can be useful if you need to analyize the the data or transform it easily.

### Getting down to the plumbing.

This module is still in beta and you should expect some things not to work. In particular, trended reports have not seen much love (though they should work), and data warehouse reports don't work at all.
This module is still in beta and you should expect some things not to work. In particular, pathing reports have not seen much love (though they should work), and data warehouse reports don't work at all.

In these cases, it can be useful to use the lower-level access this module provides through `mysuite.report.set` -- you can pass set either a key and value, a dictionary with key-value pairs or you can pass keyword arguments. These will then be added to the raw query. You can always check what the raw query is going to be with the `build` method on queries.
In these cases, it can be useful to use the lower-level access this module provides through `mysuite.report.set` -- you can pass set either a key and value, a dictionary with key-value pairs or you can pass keyword arguments. These will then be added to the raw query. You can always check what the raw query is going to be with the by simply printing the qeury.

query = network.report \
.over_time(metrics=['pageviews', 'visitors']) \
.set(dateGranularity='month')
.set({'segmentId': 'social'})
.set('name', 'my report name')
```python
query = suite.report \
.element('pages')
.metric('pageviews)
.set(anomalyDetection='month')


print query.build()
print query
```

### Running multiple reports

Expand All @@ -100,11 +169,12 @@ execution by first queueing all the reports and only _then_ waiting on the resul

Here's an example:

```python
queue = []
for segment in segments:
report = network.report \
report = suite.report \
.range('2013-05-01', '2013-05-31', granularity='day') \
.over_time(metrics=['pageviews']) \
.metric('pageviews') \
.filter(segment=segment)
queue.append(report)

Expand All @@ -113,6 +183,33 @@ Here's an example:

for report in reports:
print report.segment
print report.data['pageviews']
print report.data

```

`omniture.sync` can queue up (and synchronize) both a list of reports, or a dictionary.

### Making other API requests
If you need to make other API requests that are not reporting reqeusts you can do so by
calling `analytics.request(api, method, params)` For example if I wanted to call
Company.GetReportSuites I would do

```python
response = analytics.request('Company', 'GetReportSuites')
```

### Contributing
Feel free to contribute by filing issues or issuing a pull reqeust.

#### Build
If you want to build the module

```bash
bash build.sh
```

If you want to run unit tests

```bash
python testAll.py
```
6 changes: 6 additions & 0 deletions build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#!/bin/bash

python setup.py sdist --formats=gztar,zip
python setup.py bdist --format=gztar,zip
python setup.py bdist_egg

80 changes: 80 additions & 0 deletions build/lib/omniture/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# encoding: utf-8

from account import Account, Suite
from elements import Value, Element, Segment
from query import Query
from reports import InvalidReportError, Report, DataWarehouseReport
import os
import json
import logging.config

def authenticate(username, secret=None, endpoint=Account.DEFAULT_ENDPOINT, prefix='', suffix=''):
""" Authenticate to the Adobe API using WSSE """
#setup logging
setup_logging()
# if no secret is specified, we will assume that instead
# we have received a dictionary with credentials (such as
# from os.environ)
if not secret:
source = username
key_to_username = utils.affix(prefix, 'OMNITURE_USERNAME', suffix)
key_to_secret = utils.affix(prefix, 'OMNITURE_SECRET', suffix)
username = source[key_to_username]
secret = source[key_to_secret]

return Account(username, secret, endpoint)


def queue(queries):
""" Runs a series of reports together """
if isinstance(queries, dict):
queries = queries.values()

for query in queries:
query.queue()


def sync(queries, heartbeat=None, interval=1):
"""
`omniture.sync` will queue a number of reports and then
block until the results are all ready.

Queueing reports is idempotent, meaning that you can also
use `omniture.sync` to fetch the results for queries that
have already been queued:

query = mysuite.report.range('2013-06-06').over_time('pageviews', 'page')
omniture.queue(query)
omniture.sync(query)

The interval will operate under an exponetial decay until it reaches 5 minutes. At which point it will ping every 5 minutes
"""

queue(queries)

if isinstance(queries, list):
return [query.sync(heartbeat, interval) for query in queries]
elif isinstance(queries, dict):
return {key: query.sync(heartbeat, interval) for key, query in queries.items()}
else:
message = "Queries should be a list or a dictionary, received: {}".format(
queries.__class__)
raise ValueError(message)


def setup_logging(default_path='logging.json', default_level=logging.INFO, env_key='LOG_CFG'):
"""Setup logging configuration. """
path = default_path
value = os.getenv(env_key, None)
if value:
path = value
if os.path.exists(path):
with open(path, 'rt') as f:
config = json.load(f)
logging.config.dictConfig(config)
# Help the requests library play a little nicer
requests_log = logging.getLogger("requests")
requests_log.setLevel(logging.WARNING)

else:
logging.basicConfig(level=default_level)
Loading