Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/ticket194 #201

Merged
merged 24 commits into from
May 13, 2021
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
9fb194b
Initial implementation of autoinstrumentation. Works with the flask e…
May 8, 2021
3392c85
Resolving merge conflicts.
May 8, 2021
f86287e
Adding ability to selectively enable modules to autoinstrumentation c…
May 9, 2021
7472e8f
By default, enable instrumentation on all supported modules.
May 9, 2021
879d67d
Adding hypertrace-instrument tests to standard test suite.
May 10, 2021
e888128
Regenerating pydocs. Added autoinstrumentation example.
May 10, 2021
35a0fcb
Updating readme with autoinstrumentation instructions.
May 10, 2021
f6a649a
Updating README with clarification regarding not adding instrumentati…
May 11, 2021
6ebb408
Implement singleton for Agent class to ensure instrumentation classes…
May 11, 2021
41d52f7
Update package dependencies to set minimum package versions.
May 12, 2021
4bac59d
Updating code to fix new lint issue being reported.
May 12, 2021
08f6c1a
Correcting typo.
May 12, 2021
9af3914
A thread safe singleton implementation.
May 12, 2021
afaa6f6
Updating hypertrace_instrument run() function with comments. Broke PY…
May 13, 2021
0c0be16
sitecustomize.py cleanup.
May 13, 2021
a58419a
Adding pydoc comment back into generated file.
May 13, 2021
ac084d6
Updating flask initialization logic to correctly call instrument_app(…
May 13, 2021
bc1149c
Adding explanation of selinux commands.
May 13, 2021
a32f5d0
Adding comment back into generated file. Making argument parsing logi…
May 13, 2021
f4dcca5
Restoring Makefile to original configuration.
May 13, 2021
d9b2ae7
Removing agent.log creation.
May 13, 2021
4050b2c
Updating pydoc documentation.
May 13, 2021
3409735
Removing extra argument in FlaskInstrumentor.instrument_app().
May 13, 2021
98e8f36
docs: fixes readme.
jcchavezs May 13, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ test-integration:
cd ${TEST_DIR}/gunicorn; HT_LOG_LEVEL=${LOG_LEVEL} tox -e ${PY_TARGET}
cd ${TEST_DIR}/requests; HT_LOG_LEVEL=${LOG_LEVEL} tox -e ${PY_TARGET}
cd ${TEST_DIR}/aiohttp; HT_LOG_LEVEL=${LOG_LEVEL} tox -e ${PY_TARGET}
cd ${TEST_DIR}/docker; HT_LOG_LEVEL=${LOG_LEVEL} tox -e ${PY_TARGET}
cd ${TEST_DIR}/autoinstrumentation; HT_LOG_LEVEL=${LOG_LEVEL} tox -e ${PY_TARGET}

.PHONY: test
test: test-unit test-integration
Expand Down
15 changes: 13 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,22 @@ from hypertrace.agent import Agent
...

agent = Agent() # initialize the agent
agent.registerFlaskApp(app) # instrument a flask application
agent.registerMySQL() # instrument the MySQL client
agent.register_flask_app(app) # instrument a flask application
agent.register_mysql() # instrument the MySQL client
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice catch!

...
```

or

- Use the autoinstrumentation CLI (without any modifiation to application code)
jcchavezs marked this conversation as resolved.
Show resolved Hide resolved
```
HT_INSTRUMENTED_MODULES=flask,mysql
hypertrace-instrument python app.py
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If instrumenting a flask app, would hypertrace-instrument flask run be the more common use case?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

double tapping this comment. We should provide examples on how to launch with and without the autoinstrumentation (hypertrace-instrument)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should document starting up with hypertrace-instrument flask run to avoid confusing people accustomed to launching flask apps using the flask cli

```
By default, all supported modules are instrumented.

Note, do not attempt to instantiate a hypertrace.agent.Agent object while using hypertrace-instrument.
jcchavezs marked this conversation as resolved.
Show resolved Hide resolved

For further examples, check our [examples section](./examples)

### Configuration
Expand Down
40 changes: 20 additions & 20 deletions dev-requirements.txt
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
opentelemetry-api==1.1.0
opentelemetry-exporter-otlp==1.1.0
opentelemetry-exporter-zipkin==1.1.0
opentelemetry-instrumentation==0.20b0
opentelemetry-instrumentation-aiohttp-client==0.20b0
opentelemetry-instrumentation-wsgi==0.20b0
opentelemetry-instrumentation-flask==0.20b0
opentelemetry-instrumentation-mysql==0.20b0
opentelemetry-instrumentation-psycopg2==0.20b0
opentelemetry-instrumentation-requests==0.20b0
opentelemetry-instrumentation-grpc==0.20b0
opentelemetry-propagator-b3==1.1.0
opentelemetry-sdk==1.1.0
opentelemetry-util-http==0.20b0
google==3.0.0
opentelemetry-api>=1.1.0
opentelemetry-exporter-otlp>=1.1.0
opentelemetry-exporter-zipkin>=1.1.0
opentelemetry-instrumentation>=0.20b0
opentelemetry-instrumentation-aiohttp-client>=0.20b0
opentelemetry-instrumentation-wsgi>=0.20b0
opentelemetry-instrumentation-flask>=0.20b0
opentelemetry-instrumentation-mysql>=0.20b0
opentelemetry-instrumentation-psycopg2>=0.20b0
opentelemetry-instrumentation-requests>=0.20b0
opentelemetry-instrumentation-grpc>=0.20b0
opentelemetry-propagator-b3>=1.1.0
opentelemetry-sdk>=1.1.0
opentelemetry-util-http>=0.20b0
google>=3.0.0
pyyaml
pytest==6.2.3
protobuf==3.15.8
tox==3.23.0
pylint==2.7.4
pdoc3==0.9.2
pytest>=6.2.3
protobuf>=3.15.8
tox>=3.23.0
pylint>=2.7.4
pdoc3>=0.9.2
247 changes: 247 additions & 0 deletions docs/hypertrace/agent/autoinstrumentation/hypertrace_instrument.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,247 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, minimum-scale=1" />
<meta name="generator" content="pdoc 0.9.2" />
<title>hypertrace.agent.autoinstrumentation.hypertrace_instrument API documentation</title>
<meta name="description" content="This module implements a CLI command that be used to
autoinstrument existing pythong programs that use supported
modules." />
<link rel="preload stylesheet" as="style" href="https://cdnjs.cloudflare.com/ajax/libs/10up-sanitize.css/11.0.1/sanitize.min.css" integrity="sha256-PK9q560IAAa6WVRRh76LtCaI8pjTJ2z11v0miyNNjrs=" crossorigin>
<link rel="preload stylesheet" as="style" href="https://cdnjs.cloudflare.com/ajax/libs/10up-sanitize.css/11.0.1/typography.min.css" integrity="sha256-7l/o7C8jubJiy74VsKTidCy1yBkRtiUGbVkYBylBqUg=" crossorigin>
<link rel="stylesheet preload" as="style" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/10.1.1/styles/github.min.css" crossorigin>
<style>:root{--highlight-color:#fe9}.flex{display:flex !important}body{line-height:1.5em}#content{padding:20px}#sidebar{padding:30px;overflow:hidden}#sidebar > *:last-child{margin-bottom:2cm}.http-server-breadcrumbs{font-size:130%;margin:0 0 15px 0}#footer{font-size:.75em;padding:5px 30px;border-top:1px solid #ddd;text-align:right}#footer p{margin:0 0 0 1em;display:inline-block}#footer p:last-child{margin-right:30px}h1,h2,h3,h4,h5{font-weight:300}h1{font-size:2.5em;line-height:1.1em}h2{font-size:1.75em;margin:1em 0 .50em 0}h3{font-size:1.4em;margin:25px 0 10px 0}h4{margin:0;font-size:105%}h1:target,h2:target,h3:target,h4:target,h5:target,h6:target{background:var(--highlight-color);padding:.2em 0}a{color:#058;text-decoration:none;transition:color .3s ease-in-out}a:hover{color:#e82}.title code{font-weight:bold}h2[id^="header-"]{margin-top:2em}.ident{color:#900}pre code{background:#f8f8f8;font-size:.8em;line-height:1.4em}code{background:#f2f2f1;padding:1px 4px;overflow-wrap:break-word}h1 code{background:transparent}pre{background:#f8f8f8;border:0;border-top:1px solid #ccc;border-bottom:1px solid #ccc;margin:1em 0;padding:1ex}#http-server-module-list{display:flex;flex-flow:column}#http-server-module-list div{display:flex}#http-server-module-list dt{min-width:10%}#http-server-module-list p{margin-top:0}.toc ul,#index{list-style-type:none;margin:0;padding:0}#index code{background:transparent}#index h3{border-bottom:1px solid #ddd}#index ul{padding:0}#index h4{margin-top:.6em;font-weight:bold}@media (min-width:200ex){#index .two-column{column-count:2}}@media (min-width:300ex){#index .two-column{column-count:3}}dl{margin-bottom:2em}dl dl:last-child{margin-bottom:4em}dd{margin:0 0 1em 3em}#header-classes + dl > dd{margin-bottom:3em}dd dd{margin-left:2em}dd p{margin:10px 0}.name{background:#eee;font-weight:bold;font-size:.85em;padding:5px 10px;display:inline-block;min-width:40%}.name:hover{background:#e0e0e0}dt:target .name{background:var(--highlight-color)}.name > span:first-child{white-space:nowrap}.name.class > span:nth-child(2){margin-left:.4em}.inherited{color:#999;border-left:5px solid #eee;padding-left:1em}.inheritance em{font-style:normal;font-weight:bold}.desc h2{font-weight:400;font-size:1.25em}.desc h3{font-size:1em}.desc dt code{background:inherit}.source summary,.git-link-div{color:#666;text-align:right;font-weight:400;font-size:.8em;text-transform:uppercase}.source summary > *{white-space:nowrap;cursor:pointer}.git-link{color:inherit;margin-left:1em}.source pre{max-height:500px;overflow:auto;margin:0}.source pre code{font-size:12px;overflow:visible}.hlist{list-style:none}.hlist li{display:inline}.hlist li:after{content:',\2002'}.hlist li:last-child:after{content:none}.hlist .hlist{display:inline;padding-left:1em}img{max-width:100%}td{padding:0 .5em}.admonition{padding:.1em .5em;margin-bottom:1em}.admonition-title{font-weight:bold}.admonition.note,.admonition.info,.admonition.important{background:#aef}.admonition.todo,.admonition.versionadded,.admonition.tip,.admonition.hint{background:#dfd}.admonition.warning,.admonition.versionchanged,.admonition.deprecated{background:#fd4}.admonition.error,.admonition.danger,.admonition.caution{background:lightpink}</style>
<style media="screen and (min-width: 700px)">@media screen and (min-width:700px){#sidebar{width:30%;height:100vh;overflow:auto;position:sticky;top:0}#content{width:70%;max-width:100ch;padding:3em 4em;border-left:1px solid #ddd}pre code{font-size:1em}.item .name{font-size:1em}main{display:flex;flex-direction:row-reverse;justify-content:flex-end}.toc ul ul,#index ul{padding-left:1.5em}.toc > ul > li{margin-top:.5em}}</style>
<style media="print">@media print{#sidebar h1{page-break-before:always}.source{display:none}}@media print{*{background:transparent !important;color:#000 !important;box-shadow:none !important;text-shadow:none !important}a[href]:after{content:" (" attr(href) ")";font-size:90%}a[href][title]:after{content:none}abbr[title]:after{content:" (" attr(title) ")"}.ir a:after,a[href^="javascript:"]:after,a[href^="#"]:after{content:""}pre,blockquote{border:1px solid #999;page-break-inside:avoid}thead{display:table-header-group}tr,img{page-break-inside:avoid}img{max-width:100% !important}@page{margin:0.5cm}p,h2,h3{orphans:3;widows:3}h1,h2,h3,h4,h5,h6{page-break-after:avoid}}</style>
<script defer src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/10.1.1/highlight.min.js" integrity="sha256-Uv3H6lx7dJmRfRvH8TH6kJD1TSK1aFcwgx+mdg3epi8=" crossorigin></script>
<script>window.addEventListener('DOMContentLoaded', () => hljs.initHighlighting())</script>
</head>
<body>
<main>
<article id="content">
<header>
<h1 class="title">Module <code>hypertrace.agent.autoinstrumentation.hypertrace_instrument</code></h1>
</header>
<section id="section-intro">
<p>This module implements a CLI command that be used to
autoinstrument existing pythong programs that use supported
modules.</p>
<details class="source">
<summary>
<span>Expand source code</span>
</summary>
<pre><code class="python"># Based upon the OTel autoinstrumentation feature
&#39;&#39;&#39;This module implements a CLI command that be used to
autoinstrument existing pythong programs that use supported
modules.&#39;&#39;&#39;
import argparse
from logging import getLogger
from os import environ, execl, getcwd
from os.path import abspath, dirname, pathsep
from shutil import which

logger = getLogger(__file__)

def parse_args():
&#39;&#39;&#39;Parse CLI arguments.&#39;&#39;&#39;
parser = argparse.ArgumentParser(
description=&#34;&#34;&#34;
hypertrace-instrument automatically instruments a Python
program and runs the program
&#34;&#34;&#34;
)

parser.add_argument(&#34;command&#34;, help=&#34;Your Python application.&#34;)

parser.add_argument(
&#34;command_args&#34;,
help=&#34;Arguments for your application.&#34;,
nargs=argparse.REMAINDER,
)

return parser.parse_args()

def update_python_path() -&gt; None:
&#39;&#39;&#39;Retrieve existing PYTHONPATH&#39;&#39;&#39;
python_path = environ.get(&#34;PYTHONPATH&#34;)

# Split the paths
if not python_path:
python_path = []
else:
python_path = python_path.split(pathsep)

# Get the current working directory
cwd_path = getcwd()

# If this directory is already in python_path, remove it.
python_path = [path for path in python_path if path != cwd_path]

# If cwd is not in the PYTHONPATH, add it to the front.
if cwd_path not in python_path:
python_path.insert(0, cwd_path)

# What is the directory containing this python file?
filedir_path = dirname(abspath(__file__))

# If this directory is already in python_path, remove it.
python_path = [path for path in python_path if path != filedir_path]

# If this diretory is not in python_path, add it to the front
python_path.insert(0, filedir_path)

# Reset PYTHONPATH environment variable
environ[&#34;PYTHONPATH&#34;] = pathsep.join(python_path)

def run() -&gt; None:
&#39;&#39;&#39;hypertrace-instrument Entry point&#39;&#39;&#39;
args = parse_args()

# update PYTHONPATH env var
update_python_path()

# Get full path to the command that was passed in as an
# argument
executable = which(args.command)

# Execute the app
execl(executable, executable, *args.command_args)

if __name__ == &#39;__main__&#39;:
run()</code></pre>
</details>
</section>
<section>
</section>
<section>
</section>
<section>
<h2 class="section-title" id="header-functions">Functions</h2>
<dl>
<dt id="hypertrace.agent.autoinstrumentation.hypertrace_instrument.parse_args"><code class="name flex">
<span>def <span class="ident">parse_args</span></span>(<span>)</span>
</code></dt>
<dd>
<div class="desc"><p>Parse CLI arguments.</p></div>
<details class="source">
<summary>
<span>Expand source code</span>
</summary>
<pre><code class="python">def parse_args():
&#39;&#39;&#39;Parse CLI arguments.&#39;&#39;&#39;
parser = argparse.ArgumentParser(
description=&#34;&#34;&#34;
hypertrace-instrument automatically instruments a Python
program and runs the program
&#34;&#34;&#34;
)

parser.add_argument(&#34;command&#34;, help=&#34;Your Python application.&#34;)

parser.add_argument(
&#34;command_args&#34;,
help=&#34;Arguments for your application.&#34;,
nargs=argparse.REMAINDER,
)

return parser.parse_args()</code></pre>
</details>
</dd>
<dt id="hypertrace.agent.autoinstrumentation.hypertrace_instrument.run"><code class="name flex">
<span>def <span class="ident">run</span></span>(<span>) ‑> NoneType</span>
</code></dt>
<dd>
<div class="desc"><p>hypertrace-instrument Entry point</p></div>
<details class="source">
<summary>
<span>Expand source code</span>
</summary>
<pre><code class="python">def run() -&gt; None:
&#39;&#39;&#39;hypertrace-instrument Entry point&#39;&#39;&#39;
args = parse_args()

# update PYTHONPATH env var
update_python_path()

# Get full path to the command that was passed in as an
# argument
executable = which(args.command)

# Execute the app
execl(executable, executable, *args.command_args)</code></pre>
</details>
</dd>
<dt id="hypertrace.agent.autoinstrumentation.hypertrace_instrument.update_python_path"><code class="name flex">
<span>def <span class="ident">update_python_path</span></span>(<span>) ‑> NoneType</span>
</code></dt>
<dd>
<div class="desc"><p>Retrieve existing PYTHONPATH</p></div>
<details class="source">
<summary>
<span>Expand source code</span>
</summary>
<pre><code class="python">def update_python_path() -&gt; None:
&#39;&#39;&#39;Retrieve existing PYTHONPATH&#39;&#39;&#39;
python_path = environ.get(&#34;PYTHONPATH&#34;)

# Split the paths
if not python_path:
python_path = []
else:
python_path = python_path.split(pathsep)

# Get the current working directory
cwd_path = getcwd()

# If this directory is already in python_path, remove it.
python_path = [path for path in python_path if path != cwd_path]

# If cwd is not in the PYTHONPATH, add it to the front.
if cwd_path not in python_path:
python_path.insert(0, cwd_path)

# What is the directory containing this python file?
filedir_path = dirname(abspath(__file__))

# If this directory is already in python_path, remove it.
python_path = [path for path in python_path if path != filedir_path]

# If this diretory is not in python_path, add it to the front
python_path.insert(0, filedir_path)

# Reset PYTHONPATH environment variable
environ[&#34;PYTHONPATH&#34;] = pathsep.join(python_path)</code></pre>
</details>
</dd>
</dl>
</section>
<section>
</section>
</article>
<nav id="sidebar">
<h1>Index</h1>
<div class="toc">
<ul></ul>
</div>
<ul id="index">
<li><h3>Super-module</h3>
<ul>
<li><code><a title="hypertrace.agent.autoinstrumentation" href="index.html">hypertrace.agent.autoinstrumentation</a></code></li>
</ul>
</li>
<li><h3><a href="#header-functions">Functions</a></h3>
<ul class="">
<li><code><a title="hypertrace.agent.autoinstrumentation.hypertrace_instrument.parse_args" href="#hypertrace.agent.autoinstrumentation.hypertrace_instrument.parse_args">parse_args</a></code></li>
<li><code><a title="hypertrace.agent.autoinstrumentation.hypertrace_instrument.run" href="#hypertrace.agent.autoinstrumentation.hypertrace_instrument.run">run</a></code></li>
<li><code><a title="hypertrace.agent.autoinstrumentation.hypertrace_instrument.update_python_path" href="#hypertrace.agent.autoinstrumentation.hypertrace_instrument.update_python_path">update_python_path</a></code></li>
</ul>
</li>
</ul>
</nav>
</main>
<footer id="footer">
<p>Generated by <a href="https://pdoc3.github.io/pdoc"><cite>pdoc</cite> 0.9.2</a>.</p>
</footer>
</body>
</html>
Loading