<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

<meta charset="utf-8">

<meta http-equiv="X-UA-Compatible" content="IE=edge">

<meta name="viewport" content="width=device-width, initial-scale=1.0">

<meta name="author" content="Matt Makai">

<link rel="shortcut icon" href="theme/img/fsp-fav.png">

<title>Logging - Full Stack Python</title>

<!--[if lt IE 9]>

<link href="theme/css/f.min.css" rel="stylesheet">

<link href="//netdna.bootstrapcdn.com/font-awesome/4.1.0/css/font-awesome.css" rel="stylesheet">

<style>

.toc-indent {padding-left: 25px;}

.toc-more-indent {padding-left: 37px;}

</style>

<script>

(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){

(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),

m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)

})(window,document,'script','//www.google-analytics.com/analytics.js','ga');

ga('create', 'UA-19910497-7', 'auto');

ga('send', 'pageview');

</script> </head>

<a href="https://github.com/makaimc/fullstackpython.github.com"><img style="position: absolute; top: 0; right: 0; border: 0;" src="https://camo.githubusercontent.com/a6677b08c955af8400f44c6298f40e7d19cc5b2d/68747470733a2f2f73332e616d617a6f6e6177732e636f6d2f6769746875622f726962626f6e732f666f726b6d655f72696768745f677261795f3664366436642e706e67" alt="Fork me on GitHub" data-canonical-src="https://s3.amazonaws.com/github/ribbons/forkme_right_gray_6d6d6d.png"></a>

<div class="container">

<div class="row">

<div class="col-md-12">

<div class="logo-header-section">

<a href="/" style="text-decoration: none; border: none;"><img src="theme/img/fsp-logo.png" height="52" width="52" class="logo-image" style="padding-top: 1px;" alt="Full Stack Python logo"></a>

<span class="logo-title"><a href="/">Full Stack Python</a></span>

</div>

</div>

</div><div class="row">

<div class="col-md-8">

<h1>Logging</h1>

<p>Logging saves output such as errors, warnings and event information to

persistent storage for debugging purposes. </p>

<h2>Why is logging important?</h2>

<p>Runtime exceptions that prevent code from running are important to log to

investigate and fix the source of the problems. Informational and debugging

logging also helps to understand how the application is performing even if

code is working as intended.</p>

<h2>Logging levels</h2>

<p>Logging is often grouped into several categories:</p>

<li>Information</li>

<li>Debug</li>

<li>Warning</li>

<li>Error</li>

<p>Logging errors that occur while a web framework is running is crucial to

understanding how your application is performing. </p>

<h2>Logging aggregators</h2>

<p>When you are running your application on several servers, it is helpful

to have a monitoring tool called a "logging aggregator". You can configure

your application to forward your system and application logs to one location

that provides tools for viewing, searching, and monitoring logging events

across your cluster. </p>

<p>Another advantage of log aggregatortion tools is they allow you to set up

custom alerts and alarms so you can get notified when error rates breach a

certain threshold.</p>

<h3>Open source log aggregators</h3>

<p><a href="https://github.com/getsentry/sentry">Sentry</a> started as a Django-only

exception handling service but now has separate logging clients to cover

almost all major languages and frameworks. It still works really well for

Python-powered web applications and is often used in conjunction with other

monitoring tools. <a href="http://raven.readthedocs.org/en/latest/">Raven</a> is open

source Python client for Sentry.</p>

<p><a href="http://graylog2.org/">Graylog2</a> provides a central server for log

aggregation as well as a GUI for browsing and searching through log events.

There are libraries for most major languages, including python. Saves data

in Elasticache.</p>

<p><a href="http://logstash.net/">Logstash</a> Similar to Graylog2, logstash offers

features to programatically configure log data workflows.</p>

<p><a href="https://github.com/facebook/scribe">Scribe</a> A project written by Facebook

to aggregate logs. It's designed to run on multiple servers and scale with

the rest of your cluster. Uses the Thrift messaging format so it can be

used with any language. </p>

<h3>Hosted logging services</h3>

<p><a href="https://www.loggly.com/log-management/python-logging/?utm_source=fsp&amp;utm_medium=promo&amp;utm_campaign=march" onclick="trackOutboundLink('https://www.loggly.com/log-management/python-logging/?utm_source=fsp&amp;utm_medium=promo&amp;utm_campaign=march'); return false;">Loggly</a>

is a third party cloud based

application that aggregates logs. They have instructions for every major

language, including python. It includes email alerting on custom searches. </p>

<p><a href="http://www.splunk.com/">Splunk</a> offers third party cloud and self

hosted solutions for event aggregation. It excels at searching and data

mining any text based data. </p>

<p><a href="https://papertrailapp.com/">Papertrail</a> is similar to both

Loggly and Splunk and provides integration with S3 for long term storage.</p>

<p><a href="http://raygun.io/">Raygun</a> logs errors and provides immediate notification

when issues arise.</p>

<p><a href="https://www.scalyr.com/">Scalyr</a> provides log aggregation, dashboards,

alerts and search in a user interface on top of standard logs.</p>

<p>There is a <a href="https://www.getsentry.com/welcome/">hosted version of Sentry</a>

in case you do not have the time to set up the open source project yourself.</p>

<h2>Logging resources</h2>

<p>This

<a href="http://www.blog.pythonlibrary.org/2012/08/02/python-101-an-intro-to-logging/">intro to logging</a>

presents the Python logging module and how to use it.</p>

<p><a href="http://www.hybridcluster.com/blog/logging-storytelling/">Logging as Storytelling</a>

is a multi-part series working the analogy that logs should read like

a story so you can better understand what's taking place in your web

application.

<a href="http://www.hybridcluster.com/blog/logging-storytelling-lets-add-action/">Part 2 describes actions</a>

and

<a href="http://www.hybridcluster.com/blog/logging-storytelling-3-types/">part 3 talks about types</a>.</p>

<p><a href="https://lukasa.co.uk/2014/05/A_Brief_Digression_About_Logging/">A Brief Digression About Logging</a>

is a short post that gets Python logging up and running quickly.</p>

<p><a href="https://hynek.me/articles/taking-some-pain-out-of-python-logging/">Taking the pain out of Python logging</a>

shows a logging set up with uWSGI.</p>

<p>Django's 1.3 release brought unified logging into project configurations.

This <a href="http://www.djm.org.uk/how-to-log-file-django-13-and-above/">post shows how to set up logging</a>

in a project's settings.py file. Caktus Group also has a nice tutorial on

<a href="http://www.caktusgroup.com/blog/2013/09/18/central-logging-django-graylog2-and-graypy/">central logging with graypy and Graylog2</a>.</p>

<p><a href="http://www.caktusgroup.com/blog/2015/01/27/Django-Logging-Configuration-logging_config-default-settings-logger/">Django Logging Configuration: How the Default Settings Interfere with Yours</a>

explains a problem with the default Django logging configuration and what

to do about in your project.</p>

<h2>Logging learning checklist</h2>

<p><i class="fa fa-check-square-o"></i>

Read how to integrate logging into your web application framework. </p>

<p><i class="fa fa-check-square-o"></i>

Ensure errors and anomalous results are logged. While these logs can be stored

in <a href="/monitoring.html">monitoring</a> solutions, it's best to have your own log

storage location to debug issues as they arise to complement other monitoring

systems.</p>

<p><i class="fa fa-check-square-o"></i>

Integrate logging for system events you may need to use for debugging purposes

later. For example, you may want to know the return values on functions when

they are above a certain threshold. </p>

<h3>Logging isn't enough. How do I analyze more data about the app?</h3>

<div class="row">

<div class="col-md-4">

<div class="well select-next">

<a href="/monitoring.html" class="btn btn-success btn-full"><i class="fa fa-bar-chart-o fa-inverse fa-2x"></i></a>

<p class="under-btn">

How can I monitor my live app with tools other than just logs?

</p>

</div>

</div>

<div class="col-md-4">

<div class="well select-next">

<a href="/web-analytics.html" class="btn btn-success btn-full"><i class="fa fa-dashboard fa-2x"></i></a>

</a>

<p class="under-btn">

I want to learn more about the users of my application.

</p>

</div>

</div>

<div class="col-md-4">

<div class="well select-next">

<a href="/web-application-security.html" class="btn btn-success btn-full"><i class="fa fa-lock fa-inverse fa-2x"></i></a>

<p class="under-btn">

Something in the logs looks strange. How do I learn about security?

</p>

</div>

</div>

</div> <div id="mc_embed_signup">

<form action="//mattmakai.us2.list-manage.com/subscribe/post?u=b7e774f0c4f05dcebbfee183d&amp;id=b22335388d" method="post" id="mc-embedded-subscribe-form" name="mc-embedded-subscribe-form" class="validate" target="_blank" novalidate>

<div id="mc_embed_signup_scroll">

<h4>Sign up here to receive an email with major updates to this site and Python tutorials delivered to your inbox once a month.</h4>

<div class="row">

<div class="col-md-9">

<input type="email" value="" name="EMAIL" class="email form-control" id="mce-EMAIL" placeholder="email address" required>

<div style="position: absolute; left: -5000px;"><input type="text" name="b_b7e774f0c4f05dcebbfee183d_b22335388d" tabindex="-1" value=""></div>

</div>

<div class="col-md-3">

<div class="clear"><input type="submit" value="Subscribe" name="subscribe" id="mc-embedded-subscribe" class="btn btn-success" style="font-family: 'Helvetica Neue';"></div>

</div>

</div>

</div>

</div>

<div class="col-md-offset-1 col-md-3" id="sidebar">

<div class="panel panel-success">

<div class="panel-heading">

<h3 class="panel-head">Sponsored By</h3>

</div>

<div class="panel-body">

<a href="https://www.loggly.com/log-management/python-logging/?utm_source=fsp&amp;utm_medium=promo&amp;utm_campaign=march"><img src="theme/img/sponsored/loggly.jpg" alt="Loggly with Beaver logo" width="100%"></a>

<p style="font-size: .8em; margin-top: 10px;">Want to improve your Python

web app's logging performance? I highly recommend

<a href="https://www.loggly.com/log-management/python-logging/?utm_source=fsp&amp;utm_medium=promo&amp;utm_campaign=march" onclick="trackOutboundLink('https://www.loggly.com/log-management/python-logging/?utm_source=fsp&amp;utm_medium=promo&amp;utm_campaign=march'); return false;">signing up for a free Loggly account</a>.

</p>

</div>

<div class="panel panel-success">

<div class="panel-heading">

<h3 class="panel-head"><a href="/table-of-contents.html" style="color: #fff;">Table of Contents</a></h3>

</div>

<div class="list-group">

<a href="/introduction.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Introduction</a>

<a href="/why-use-python.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Why Use Python?</a>

<a href="/best-python-resources.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Best Python Resources</a>

<a href="/best-python-videos.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Best Python Videos</a>

<a href="/development-environments.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Development Environments</a>

<a href="/vim.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Vim</a>

<a href="/emacs.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Emacs</a>

<a href="/generators.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Generators</a>

<a href="/comprehensions.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Comprehensions</a>

<a href="/web-frameworks.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Web Frameworks</a>

<a href="/django.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Django</a>

<a href="/flask.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Flask</a>

<a href="/bottle.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Bottle</a>

<a href="/pyramid.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Pyramid</a>

<a href="/morepath.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Morepath</a>

<a href="/other-web-frameworks.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Other Web Frameworks</a>

<a href="/web-design.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Web Design</a>

<a href="/cascading-style-sheets.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Cascading Style Sheets</a>

<a href="/javascript.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>JavaScript</a>

<a href="/websockets.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>WebSockets</a>

<a href="/web-application-security.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Web Application Security</a>

<a href="/databases.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Databases</a>

<a href="/no-sql-datastore.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>NoSQL Data Stores</a>

<a href="/application-programming-interfaces.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Application Programming Interfaces</a>

<a href="/api-integration.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>API Integration</a>

<a href="/api-creation.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>API Creation</a>

<a href="/deployment.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Deployment</a>

<a href="/servers.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Servers</a>

<a href="/platform-as-a-service.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Platform-as-a-service</a>

<a href="/operating-systems.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Operating Systems</a>

<a href="/web-servers.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Web Servers</a>

<a href="/wsgi-servers.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>WSGI Servers</a>

<a href="/source-control.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Source Control</a>

<a href="/application-dependencies.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Application Dependencies</a>

<a href="/static-content.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Static Content</a>

<a href="/task-queues.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Task Queues</a>

<a href="/configuration-management.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Configuration Management</a>

<a href="/continuous-integration.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Continuous Integration</a>

<a href="/logging.html" class="list-group-item smaller-item active" style='font-family: "Helvetica Neue",sans-serif;'>Logging</a>

<a href="/monitoring.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Monitoring</a>

<a href="/web-analytics.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Web Analytics</a>

<a href="/docker.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Docker</a>

<a href="/caching.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Caching</a>

<a href="/code-metrics.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Code Metrics</a>

<a href="/what-full-stack-means.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>What "Full Stack" Means</a>

<a href="/change-log.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Change Log</a>

<a href="/future-directions.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>Future Directions</a>

<a href="/about-author.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>About the Author</a>

</div>

</div>

<div class="panel panel-success">

<div class="panel-heading"><h3 class="panel-head">Logging</h3></div>

<div class="panel-body">

Major updates are tweeted via

<a href="https://twitter.com/fullstackpython">@fullstackpython</a>.

<hr/>

Need more detailed tutorials than you see here?

<a href="/email.html">Sign up to receive an email when that content is created.</a>

</div>

</div>

<hr/>

<div class="footer pull-right">

<a href="http://www.mattmakai.com/" class="underline">Matt Makai</a>

2015

</div>

</div>

<script type="text/javascript">

setTimeout(function(){var a=document.createElement("script");

var b=document.getElementsByTagName("script")[0];

a.src=document.location.protocol+"//script.crazyegg.com/pages/scripts/0016/5846.js?"+Math.floor(new Date().getTime()/3600000);

a.async=true;a.type="text/javascript";b.parentNode.insertBefore(a,b)}, 1);

</script><script type='text/javascript'>

var trackOutboundLink = function(url) { ga('send', 'event', 'outbound', 'click', url, {'hitCallback': function () { document.location = url; } }); }