Unlock Jupyter Notebook Power: Shell Commands, Magic, Logging & Seaborn Tricks

Preface

In the previous article we introduced a Jupyter beginner tutorial; this article presents additional Jupyter Notebook usage tips.

The main topics covered are:

Introducing some basic shell commands and convenient magic commands, including debugging, timing, and executing multiple languages.

Exploring logging, macros, running external code, and Jupyter extensions.

Enhancing Seaborn charts, running commands from the command line, and using databases.

Shell Commands

In a notebook you can run shell commands by starting a line with ! inside a code cell. This is useful for data or file handling and managing Python packages.

You can also embed Python variables into shell commands using $:

Because commands prefixed with ! are discarded after execution, commands like cd have no lasting effect. IPython magic commands provide a workaround.

Basic Magic Commands

Magic commands are built‑in IPython commands for specific tasks. They resemble Unix commands but are implemented in Python. There are two types:

Line magics (prefixed with %)

Cell magics (prefixed with %%)

Use %lsmagic to list available magics, which are divided into line and cell categories.

Line magics start with %, cell magics with %%. The ! syntax is a special form of shell magic; commands like %cd, %alias, and %env can be used instead of plain cd.

Autosaving

The %autosave command sets the notebook autosave interval (seconds). %autosave 60 Output:

Autosaving every 60 seconds

Displaying Matplotlib Figures

The line magic %matplotlib (often with the inline argument) enables Matplotlib figures to be shown inside the notebook. %matplotlib inline Place this at the top of the notebook before importing Matplotlib.

Timing Execution

Use %time and %timeit to measure execution time. Both have line and cell forms.

%timeit

runs the code multiple times and reports an average; you can control repetitions with -n.

Executing Different Programming Languages

Although the kernel is Python 3, magic commands allow execution of other languages such as HTML, LaTeX, Ruby, Markdown, JavaScript, and R.

Configuring Logging

Import the logging module to customize error output. Logging writes to stderr, while print writes to stdout.

Example configuration to show INFO and DEBUG messages:

logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
logging.info('This is some information')
logging.debug('This is a debug message')

Custom formatters and handlers can be added:

handler = logging.StreamHandler()
handler.setLevel(logging.DEBUG)
formatter = logging.Formatter('%(levelname)s: %(message)s')
handler.setFormatter(formatter)
logger.handlers = [handler]
logging.error('An error')
logging.warning('An warning')
logging.info('An info')

Logs can also be written to a file using FileHandler:

handler = logging.FileHandler(filename='important_log.log', mode='a')

Extensions

Jupyter has many community extensions. Install them with:

pip install ipython-sql
pip install jupyter_contrib_nbextensions
jupyter contrib nbextension install --user
jupyter nbextension enable spellchecker/main
jupyter nbextension enable codefolding/main

Enhancing Seaborn Charts

Install Seaborn if needed:

!pip install seaborn

Load data and create a basic scatter plot:

%matplotlib inline
import matplotlib.pyplot as plt
import seaborn as sns
data = sns.load_dataset("tips")
plt.scatter(data.total_bill, data.tip)

Apply Seaborn styles such as darkgrid:

sns.set(style="darkgrid")
plt.scatter(data.total_bill, data.tip)

Use sns.scatterplot for richer plots, adding hue and style dimensions:

sns.scatterplot(x="total_bill", y="tip", hue="smoker", data=data)

sns.scatterplot(x="total_bill", y="tip", hue="size", style="smoker", data=data)

Macros

Save a code cell as a macro with %macro and %store, then reload it with %store -r.

name = 'Ben'
__hello_world

Hello, Ben!

Running External Code

Load and run external .py files with %load and %run.

%load imports.py

Create a script triangle_hist.py and execute it:

import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
sns.set(style="darkgrid")
if __name__ == '__main__':
    h = plt.hist(np.random.triangular(0, 5, 9, 1000), bins=100, linewidth=0)
    plt.show()

Script Execution

Convert notebooks to other formats with jupyter nbconvert. Example to PDF:

jupyter nbconvert --to pdf notebook.ipynb

Add --execute to run the notebook during conversion, and --allow-errors to keep errors in the output.

jupyter nbconvert --to pdf --execute --allow-errors notebook.ipynb

Using Databases

Install ipython-sql and load the extension:

pip install ipython-sql
%load_ext sql

Connect to a SQLite database:

%sql sqlite://

Run SQL queries using cell magic %%sql or line magic %sql. Example connection string format:

dialect+driver://username:password@host:port/database

Further examples are available at the ipython‑sql GitHub repository.

Conclusion

The original article omitted some sections such as script execution details and custom notebook styling; the core code examples have been retained.

Relevant links:

https://github.com/catherinedevlin/ipython-sql

https://github.com/ipython-contrib/jupyter_contrib_nbextensions

https://github.com/mwaskom/seaborn-data

https://stackoverflow.com/a/14411126/604687

https://stackoverflow.com/questions/582336/how-can-you-profile-a-python-script/582337#582337

https://nbconvert.readthedocs.io/en/latest/install.html#installing-nbconvert

https://stackoverflow.com/a/52913424/604687