Lightning talk from Pycon Sweden 2022.

1. MY
MY
LOGS
LOGS
HAVE SOMETHING TO TELL YOU 🦉
HAVE SOMETHING TO TELL YOU 🦉
2

2. WHAT ARE LOGS?
WHAT ARE LOGS?
Troubleshooting, Auditing, Profiling, Statistics
Troubleshooting, Auditing, Profiling, Statistics
Logging ≠ Observability ≠ Output
Logging ≠ Observability ≠ Output
3

3. WHAT ARE LOGS?
WHAT ARE LOGS?
Troubleshooting, Auditing, Profiling, Statistics
Troubleshooting, Auditing, Profiling, Statistics
Logging ≠ Observability ≠ Output
Logging ≠ Observability ≠ Output
Logging
Logging
Output meant for postmortem troubleshooting.
Output meant for postmortem troubleshooting.
4

4. THE BASICS
THE BASICS
5

5. THE BASICS
THE BASICS
import logging
logger = logging.getLogger(__name__)
user = "cooper"
town = "Twin Peaks"
logger.info("%s entering %s", user, town,
extra={"user": {"name": user}})
6

6. Automatic Extra Data
Automatic Extra Data
log = LoggerAdapter(
logger, extra={"user": {"name": user}}
)
logger.info("%s entering %s", user, town)
7

7. Global Extra Data
Global Extra Data
# utils/logs.py
local = threading.local()
def set_user(user: dict) -> None:
local.user = user
class ExtraDataFilter(logging.Filter):
"""
Adds global extra data to all log records
"""
def filter(self, record):
if user := getattr(local, "user", None):
record.user = user
8

8. try:
pour_coffee(cup)
except Exception:
logger.exception("Something strange occurred with %s", cup)
9

9. BEST PRACTISES
BEST PRACTISES
10

10. BEST PRACTISES
BEST PRACTISES
Be careful of Sensitive Information
Log relevant events *
Consider the reader. *
Let the logging library construct the message *
11

11. BEST PRACTISES
BEST PRACTISES
LOG RELEVANT EVENTS
LOG RELEVANT EVENTS
12

12. BEST PRACTISES
BEST PRACTISES
LOG RELEVANT EVENTS
LOG RELEVANT EVENTS
Log on changes
Log on strangeness
Business events and decisions
Network calls
Decision rationale
HTTP world: POST PUT DELETE.
HTTP world: POST PUT DELETE.
13

13. BEST PRACTISES
BEST PRACTISES
CONSIDER THE READER
CONSIDER THE READER
14

14. BEST PRACTISES
BEST PRACTISES
CONSIDER THE READER
CONSIDER THE READER
Logs are read in multitues.
Logs are read in multitues.
Keep them Short.
Keep them Self-contained.
Keep them Consistent.
Add the necessary data.
15

15. BEST PRACTISES
BEST PRACTISES
FORMATTING
FORMATTING
Do let the logging library construct the message.
Do let the logging library construct the message.
Do use time in seconds, unless otherwise specified.
Do use time in seconds, unless otherwise specified.
logger.info("%s entering %s", user, town)
16

16. BEST PRACTISES
BEST PRACTISES
FORMATTING
FORMATTING
Do add global contextual information, e.g. tracing IDs.
Do add global contextual information, e.g. tracing IDs.
Do rename and filter the default
Do rename and filter the default extra
extra data for
data for
better readability.
better readability.
Do define your
Do define your extra
extra schema,
schema,
with
with !
!
Elastic Common Schema
Elastic Common Schema
17

17. BEST PRACTISES
BEST PRACTISES
ANALYSIS
ANALYSIS
18

18. BEST PRACTISES
BEST PRACTISES
ANALYSIS
ANALYSIS
ELK stack with ElasticSearch and Kibana
ELK stack with ElasticSearch and Kibana
= Fantastic. ✨🚀
= Fantastic. ✨🚀
Sentry / LogRocket.
Sentry / LogRocket.
19

19. LIBRARIES
LIBRARIES
python-logstash-formatter
ecs-logging-python
loguru
structlog
20

20. 21