Learn coding in Python, Go and Rust from Serdar Yegulalp, software dev specialist and senior writer at InfoWorld.
In previous videos we covered Cython (for translating Python into C for speed), Numpy (for working with numerical arrays at scale), and parallelism in Cython (for doing work in multiple threads). In this video we look at how all three of these things can be used at once: writing Cython that runs in parallel for operating on Numpy arrays. We'll also cover how to ensure Cython uses Numpy arrays correclty, and why parallelism with Numpy arrays doesn't always result in linear speedups.
Many different data libraries in Python work with dataframes – Polars, Pandas, DuckDB, and so on. But they all sport highly dissimilar APIs and feature sets, and some have features the others don't. Ibis provides a common, unified front end for over a dozen Python dataframe libraries – a common way to read data, query it, pass dataframes between different processing engines, and report the results. This video shows an example of how this works using a single common dataframe derived from a CSV, using DuckDB, Polars, and Ibis's native dataframe processing.
Asynchronous programming in any language can be tricky, and Python is no exception. The AnyIO library simplifies many common patterns in async programming in Python, such as how to start async tasks or manage them. AsyncIO also adds many behaviors and features for async not found in Python's standard library, like performing asynchronous work with files.
Analytics databases, or column-oriented DBs, speed up analytics jobs on big datasets. But most such DBs are big products with a lot of setup and management overhead. DuckDB bucks that trend: it's a single, self-contained executable and library that brings SQL-query-powered analytics DB functionality to most any programming language, workflow, or data trove. This video shows how you can use DuckDB with Python to work with big, multi-megabyte JSON or CSV datasets, and query the data using conventional SQL syntax.
As WebAssembly gains in popularity, one possible application is to embed a WASM runtime into applications to extend the app's functionality. Extism lets you write WASM extensions for your own applications, creating in effect a WASM-powered plugin system for your apps without the overhead of embedding a full language runtime like Lua. This video shows the basics of how to write a WASM add-on in AssemblyScript, and add it to a Python application, all in a few lines of code.
Cython's used to take existing Python code and translate it to C for speed. It can also be used to make some operations on C types faster by performing them in parallel, and without using the overhead of Python's threading or multiprocessing mechanisms. Instead, you can write Cython code that makes use of the industry-standard OpenMP interface for writing parallel-processing code. This video goes through the basics of using OpenMP in Cython, using the "prange" object, and gives you an idea of what's involved to make operations parallel.
If you've ever needed to compile something from C/C++ source in Microsoft Windows, odds are you've also been aggravated by how complex it can be to set up the Visual Studio C++ Build Tools. Turns out there's a simple, one-command way to do it using the Winget package manager, with minimal babysitting and fuss. Watch this video to learn how to install the Build Tools from the command line, and how to customize the setup to include more than the basic components if needed.
Writing a web app in Python isn't hard, as long as you understand the frontend components – HTML, CSS, and JavaScript. It's harder when you don't know those things, especially if you want to write a Python app that lets users freely explore data through interactive components. Streamlit lets you write web apps in pure Python, and outfit them with a wide-ranging gallery of interactive data exploration widgets. This video shows the basics of Streamlit in action, demonstrates how Streamlit apps are typically written and how they behave, and also walks through a few caveats worth knowing about.
Normally, Visual Studio Code is a text editor, but its panoply of add-ons can transform it into a versatile development tool for working with many aspects of software development. In this video we look at some examples of how VS Code can be made into a database exploration and workbench tool by way of various free add-ons.
The creators of the "ruff" linter have a new Rust-powered tool for Python – an all-in-one project creation and management tool named "uv". See in this video how "uv" performs the same jobs as "pip" and many other Python tools, but a great deal faster. But how much of a future is there for Python tools written in Rust?
Got ambitions about getting into game development? This video walks you through some specific steps to take towards that goal: laying a proper foundation with knowing software development generally; how to not let ambitions to create your dream game derail things; how to choose the kinds of games to make; and more.
The PyScript project offers the promise of being able to run Python programs as-is in a web page, as transparently as one runs JavaScript programs. The project's been through some major revisions since we last looked at it, as it's now somewhat easier to set up and work with. Watch as we set up a few basic Python programs in PyScript, which interact with the web page they're embeded in, or even use an embedded in-browser console for interactivity with the user.
Video games have become a massive and lucrative market, bigger even than the movie industry. If you've considered video game development as a career move, it's worth knowing what kind of prerequisites are involved. In this video we look at a few things you'll need to know before considering video game development as a career – not just skills, but habits and expectations.
All Python developers write loops in their code. But sometimes the built-in looping mechanisms aren't enough – sometimes you want to write loops that handle sophisticated scenarios. What if you want to loop through multiple things of different lengths together? Or loop through all the possible combination of some things? Learn how the itertools module in Python lets you do all these things and more, without having to write your own custom code to do them.
Your Python programs shouldn't be black boxes – they should be able to tell you about what's going on inside them as they run. The logging module for Python gives you convenient ways to write log messages to the console, to use different levels of verbosity for logging, and to configure defaults for logging behavior. Learn the basics of the logging module in this video, with a few easy examples to get you started.
Python projects that use modern tooling and standards work best when the files in the project repository follow a certain layout. Learn in this video how to organize a Python project's files and directories, including: using the pyproject.toml file to describe a project's metadata, setting up .gitignore for Python projects, and where to place tests or documentation.
Developers who use Microsoft Windows as their main platform often grouse about how Windows's malware protection can interfere with their work. The new Dev Drive feature in Windows 11 lets you create a special volume, either on a physical disk or a virtual drive, where your files are only minimally scanned for malware. Learn how to set up and use Dev Drive in this video, and what scenarios it's best for.
A new API introduced in Python 3.12 provides developers with efficient, low-impact ways to hook events in Python code, monitor behavior, and write powerful profiling and coverage tools. Watch this video to see a quick tour of the new API and how to use it, with an example where we write a hook function that fires on every function call in an application. For further information on the new API, see PEP 669: https://peps.python.org/pep-0669/
Python's linting and code formatting tools are typically written in Python, and can slow down significantly on large code bases. Ruff, a new linter and code formatter for Python, is written in Rust, and runs many times faster than comparable Python tools. But it also adheres closely to existing Python standards for code-checking and formatting. This video shows Ruff in action, and how it stacks up against existing Python tools for speed.
Cython compiles Python to C for high performance, and Cython modules can be redistributed with your code in a Python package -- a source distribution or "wheel" file -- so others can use it, too. You can deliver both the generated C source code, and a precompiled binary for a specific platform, for the widest possible re-use. Learn in this video how to set up your project when Cython code is present, and what tools you'll need to build wheels and source distributions from it. Also be sure to check out some of our other videos about Cython: https://www.infoworld.com/video/116719/cython-3-0-compiling-python-to-c-the-next-generation https://www.infoworld.com/video/112519/cythons-new-pure-python-syntax-faster-python-made-easier
WebAssembly has been described as the future of the web, and so much more besides. But for a long time it's been a little hard to work with because of its toolchain. In this video we'll look at AssemblyScript, a JavaScript-like language, based on Microsoft's TypeScript, that makes writing and deploying WebAssembly modules as easy as writing JavaScript itself.
How do you do more than one thing at a time in Python? In this video we lay out the two main ways we can do more than one thing at a time -- parallelism, and concurrency -- and show how Python has three mechanisms for doing those things. Learn how threading, async, and multiprocessing all make it possible to juggle multiple tasks or run them side by side.
Writing interactive web apps requires JavaScript, and often a lot of it. Too much of the time spent developing a web app gets eaten by creating many of the same behaviors across sites, like fetching content from an endpoint and inserting it into a page. The Htmx JavaScript library makes it easy to add common interactive behaviors on web pages, often with nothing more than a couple of custom HTML tags. This video shows a few basic examples, from type-to-search input boxes to infinite-scroll displays.
NumPy, the Python library for fast array and matrix math, is a staple presence in the data science world. A new major revision of NumPy, version 2.0, is on the horizon. Here's what breaking changes to expect with this new version, what new features might be offered, and how to prepare your projects for them.
Want an all-in-one way to create, manage, and package Python projects -- which even automates fetching and setting up Python itself? Rye, an experimental new project written in Rust, provides a command-line tool for obtaining Python runtimes, setting up new projects, adding requirements, publishing to PyPI, and many other common tasks that normally have to be done manually. In this video, you'll see a quick example of Rye in action, and what kinds of tasks it can automate.
Mojo, a newly developed language aimed at machine learning and data science tasks, promises C++- and Rust-like speeds with the syntax of Python. Previously, the only way to try Mojo was through its online portal. Now the language's developer, Modular, has released a locally installable version of Mojo, although it's currently only available for Linux. In this video we take a simple walkthrough of its basic operations, running in WSL on Windows, and see how it interoperates with Python and compiles to native binaries.
Unlike the various breeds of Linux, Microsoft Windows hasn't really had a package management mechanism, except for ad-hoc third party offerings or the proprietary Microsoft Store. Winget offers a Microsoft-authored open source mechanism for managing, retrieving, and updating whole sets of software in Windows. Learn the basics of how Winget works, along with examples of how third-party apps can be built atop Winget for even easier package management solutions, too.
The Go programming language is normally used with its own libraries and packages, but it can interface with the world of C libraries and programs, too. Learn in this video how to use the cgo package to connect with C libraries, interface with them through your Go programs, and handle a few common issues with crossing the boundaries between the two languages.
Cython makes it easy to create C extensions for Python, using the Python syntax you already know. It also lets you create convenient Python wrappers for C libraries using their header files. Learn in this video how to create Python function bindings to a C library, by writing a Cython module and using its external function binding syntax.
CPython's Global Interperter Lock, or GIL, has long prevented it from being a truly parallel, multicore language. A new proposal, PEP 703, describes a roadmap to a new design for CPython that can use multithreading to properly unlock parallelism. Watch in this video as we use a prototype build of the "no-GIL" edition of Python to demonstrate how much faster code can be if this proposal takes off. Read more here about PEP 703 and what it promises: https://www.infoworld.com/article/3704248/python-moves-to-remove-the-gil-and-boost-concurrency.html
The Cython project compiles Python to C -- not just to make code faster, but also to wrap external C libraries easily, and make it easy to write C extensions for Python generally. Cython 3.0 is a major overhaul of the project. Learn in this video what's new about it, why it's worth upgrading your Cython code to use its features, and some pitfalls to avoid along the way. I talk more about the basics of Cython here: https://www.infoworld.com/article/3250299/what-is-cython-python-at-the-speed-of-c.html More here about the CPython limited API, and other future changes to CPython: https://www.infoworld.com/article/3694512/python-312-faster-leaner-more-future-proof.html
Moden Python projects use pyproject.toml files to describe a project's metadata, dependencies, and other valuable attributes. Learn in this video the basics of how pyproject.toml works -- how it can be used to describe a simple project, install its dependencies, and define other things such as command-line scripts used with the project.
Python's Django framework lets you spin up full-featured websites with database backends, user management systems, and many other features. In this fourth episode of a series, we'll explore how to take the basic message board app we've been using as an example, and add the ability for user information to be associated with a given post. We'll also show how to allow users to log in using Django's own utilities for processing logins. You can access the code used in this example here: https://github.com/syegulalp/django-2023/releases/tag/part-4
The Python embeddable redistribution (for Windows only) gives you a minimal, self-contained Python runtime that runs in a single directory. Learn how to use the embeddable redistribution to package up and distribute simple Python applications for re-use, and how to install third-party packages in the embeddable with pip if you need them. The Python embeddable redistribution can be downloaded from Python.org (it's listed as the "Windows embeddable package" with a given release version).
The Python buffer protocol provides a mechanism for directly accessing regions of memory wrapped by Python objects. Until now you couldn't create a Python class that exposed the buffer protocol directly, but Python 3.12 adds class methods to do just that. Learn in this video how to create classes that expose the buffer protocol, and how to manage the resulting memory views. Part 1: https://youtu.be/AHtChZg2cFA
Python 3.12 is out in beta now and worth trying out, with many new features that aid day-to-day productivity. In this video we'll look at two big new improvements: better, more robust error messages; and improvements to Python's f-string formatting system.
Microsoft Visual C++ (MSVC for short) is the default C/C++ compiler for Windows, but it isn't included by default in that OS -- a common developer's complaint. And installing Microsoft Visual Studio, which includes MSVC, can be overkill if you just need MSVC by itself. In this video we'll walk through the fast way to install just the MSVC build tools, typically all you need to compile C/C++ in Windows from the command line.
Mojo, a new language designed to be a more powerful Python, is now available in a limited preview. In this video, we take a look at Mojo's syntax and design, and explore the ways it aims to unite convenient programming with the speed and power needed for data science and machine learning.
How much thought do you give to the fonts you use in your code editor? Here's a look at some of the most popular and versatile fonts for working with code, from basic all-around typefaces to advanced, highly hackable and customizable fonts with web-based tooling.
Python's Django framework lets you spin up full-featured websites with database backends, user management systems, and many other features. In this second episode of a series, we'll examine how to create a basic route for a Django app, and how to set up the way data for the app is represented in Django's database system. Part 1: https://youtu.be/055sCJfk0f0
Python's Django framework lets you spin up full-featured websites with database backends, user management systems, and many other features. In this first episode of a series, we'll walk through the basics of setting up a Django project -- creating the project, defining apps within it, and preparing the database and user account system.
The Poetry utility for Python provides a high-level way to set up Python projects, manage their dependencies, and redistribute them through PyPI, all through a single handy command-line interface. This video provides a quick tour of Poetry's basic functionality for a sample project, including working with Poetry's managed virtual environments in an instance of Visual Studio Code.
Cython lets you write C extension modules for Python using Python itself. A common use for this is to take Python code and rewrite it in C for speed. Learn in this video what scenarios are the best fit for using Cython to speed up Python, and how Cython's type-decorating syntax helps you achieve this.
Regular expressions give you a powerful system for matching patterns against text. But there are cases where a regex (as it's called) can be misused, where it's not the only tool you should be using, or when it's just not the right tool, period. Learn about 4 cases where regex isn't a great fit, when it needs a fine-grained approach, or when you need something else altogether.
Tired of having your Python projects littered with .pyc cache files and __pycache__ directories? This video teaches you about a little-used but powerful Python feature that lets you keep all your Python cache files in a single directory, away from your project directories and under central control.
The Codon project offers a highly Python-compatible language, built from the ground up, that compiles to native machine code for speed. It's not a drop-in replacement for Python, but it promises some significant speedups for math-and-stats applications. Could Codon's approach be how Python gets faster in general? Or is Codon more of a complement to the existing Python ecosystem rather than a long-term replacement? Watch and learn more.
Want to create cross-platform apps with web UIs, but don't like Electron's sheer size? In this video we'll check out Tauri, a framework for building cross-platform, web-powered apps with Rust. Tauri apps have a fraction of the size of Electron apps, and the Tauri toolset lets you use anything from vanilla HTML to frameworks like Svelte for your app's frontend. Watch us build a simple Tauri app, modify its HTML components without recompiling, and create an .MSI installable package from it.
Go's internal assembly language is generated from Go source code as an intermediate step towards creating a machine-specific binary. You can dump out the generated assembly and inspect it using Go's tooling, and use the generated code to get an idea of what kinds of optimizations are being applied to your code. Learn the basics of how this works in this video.
Rust has a number of web frameworks. Actix Web is the most popular and widely used. Learn in this video how a simple web service can be set up in Actix, and how Actix makes use of Rust's async programming features for maximum performance.
A common frustration for the Python developer is circular imports -- when two modules each try to import something from the other simultaneously. Learn in this video how this problem arises in Python applications, and several methods you can use to fix or avoid it entirely.
Sponsored Links