Python4DataScience
LightNews LightNews
banner
python4data.science
Python4DataScience
@python4data.science
4.8K followers 4 following 110 posts
Teaching materials for the cusy training courses on a Python-based data science workflow: https://cusy.io/seminars
Posts Replies Media Videos
python4data.science
It seems that HTTPX is no longer being developed. So we switched to HTTPX2: www.python4data.science/en/latest/da...
#Python #HTTPX
HTTPX2
HTTPX2 is an HTTP client that can be used seamlessly as a replacement for HTTPX and simplifies HTML requests compared to the Python standard library’s urllib.request. Installation: You can easily i...
www.python4data.science
October 12, 2026 at 12:41 PM
Reposted by Python4DataScience
cusy.io
🧵1/2
Just in time for the release of #Python 3.15, we've updated our workshop tutorials with
• lacy imports: python-basics-tutorial.readthedocs.io/en/latest/mo...
• frozendict: python-basics-tutorial.readthedocs.io/en/latest/ty...
…
We love Python
October 5, 2026 at 5:59 AM
Reposted by Python4DataScience
veit-schiele.de
In this week’s workshop, we applied TDD to legacy software. We also wrote down what lessons we can take away for other projects: python-basics-tutorial.readthedocs.io/en/latest/te...
#TDD #Legacy #Software #DX
Legacy software
There may be various reasons for wanting to modify existing software: to add a feature, to fix a bug, to improve the software architecture (→ Refactoring), to improve resource usage. If an existing...
python-basics-tutorial.readthedocs.io
October 2, 2026 at 12:02 PM
python4data.science
Git 2.56 introduced a number of exciting features that improve the way we work with merges: 'git add --resolved' and 'git merge-base'.
www.python4data.science/en/latest/pr...
www.python4data.science/en/latest/pr...
#Git #DeveloperExperience #DX
Git branches
$ git branch [-a] [-l " GLOB_PATTERN"], shows all local branches in a repository.--a, also shows all removed branches.,,-l, restricts the branches to those that correspond to a specific p...
www.python4data.science
September 29, 2026 at 2:16 PM
python4data.science
Having first taken a closer look at CSVW for table dialects and schemas, we have now examined data resource and data package formats in detail, so that we can publish and cite data more effectively: www.python4data.science/en/latest/pr...
#Python #DataScience
Cite data
Similar to the way in which researchers routinely provide bibliographical references to other academic sources, data should also be cited. Although data is frequently shared, it is not always cited...
www.python4data.science
September 23, 2026 at 12:04 PM
python4data.science
CSVW is a metadata standard for CSV files. If it becomes widely adopted, it will make processing CSV files so much better: www.python4data.science/en/latest/da...
#Python #DataScience
CSV
Overview:,,, Data structure support,--, CSV is used to store tabular data, but unlike other serialisation formats reviewed here, it’s not suitable for (nested) objects.,, Standardisation,--, CSV is...
www.python4data.science
September 21, 2026 at 12:13 PM
python4data.science
We have significantly revised the section on data validation and cleansing and expanded it to include data quality criteria and processes: www.python4data.science/en/latest/cl...
#Python #DataScience
Cleaning and validating data
‘Garbage in, garbage out’ is a stark reminder that it is almost impossible to draw meaningful insights from poor-quality data. Whilst there are highly regulated, safety-critical sectors in which da...
www.python4data.science
September 18, 2026 at 1:34 PM
python4data.science
We have updated the section on text comparison – featuring difflib, TheFuzz and textacy: www.python4data.science/en/latest/cl... #Python
String comparisons
difflib: You can compare strings using the get_close_matches option of Python’s built-in difflib library. If you want to specify the minimum match score required for a suggestion to be displayed, y...
www.python4data.science
September 3, 2026 at 6:03 AM
python4data.science
At first, it can sometimes be difficult to imagine a practical use for decorators. We will therefore now present a range of different examples: python-basics-tutorial.readthedocs.io/en/latest/fu...
#Python #Decorators
Decorators
Functions can also be passed as arguments to other functions, and the results of other functions can be returned. For example, it is possible to write a Python function that takes another function ...
python-basics-tutorial.readthedocs.io
August 18, 2026 at 6:10 AM
python4data.science
Narwhals provides a lightweight and extensible compatibility layer between various DataFrame libraries: www.python4data.science/en/latest/pe...
#Python #pandas #DataFrame
Parallelise pandas
In Enhancing performance, some possibilities are described for improving the performance of pandas. However, there are also special libraries that can parallelise the processing of data frames. cuD...
www.python4data.science
August 11, 2026 at 11:58 AM
python4data.science
We have expanded the section on mocks for testing and added spies: python-basics-tutorial.readthedocs.io/en/latest/te...
#Python #Testing
Mock
In this chapter, we will test the CLI. To do this, we will use the mock library, which has been included as part of the standard Python library since Python 3.3 under the name unittest.mock. Object...
python-basics-tutorial.readthedocs.io
August 5, 2026 at 12:46 PM
python4data.science
🧵 1/2 Attacks on the software supply chain are increasing rapidly. We have therefore updated our guide on how to make your projects more secure, adding answers to the following questions:
Security
As of July 2026, PyPI hosts over 750,000 packages, and this number is growing daily. An average Python project includes dozens of additional dependencies – packages that you have never explicitly c...
www.python4data.science
August 3, 2026 at 6:13 AM
python4data.science
Did you know that with shot-scraper video you can explain your user interface using a storyboard: python-basics-tutorial.readthedocs.io/en/latest/do...
#Python #Docs #Video
shot-scraper
shot-scraper is a tool to automate the process of updating screenshots. Installation: Use: shot-scraper can be used in several ways …for single screenshots on the command line: …or with additional ...
python-basics-tutorial.readthedocs.io
July 24, 2026 at 7:29 AM
python4data.science
Software Bill of Materials (SBOMs) are commonly used to check software and its dependencies for vulnerabilities using CVE and OSV. We have described how to create these for Python packages here: python4data.science/en/latest/pr...
#Python #SBOM #Vulnerability
Software Bill of Materials (SBOM)
A Software Bill of Materials (SBOM) is a document used to exchange information about software and its composition. This format is primarily used in the security field to check software and its depe...
python4data.science
July 22, 2026 at 5:50 AM
python4data.science
Specification-Driven Development (SDD) was developed to improve the predictability and maintainability of agentic software development. In particular, we are continuing to give OpenSpec a chance: agentic-software-engineering.readthedocs.io/en/latest/pr...
#CodingAgent #SDD #DX
Spec-Driven Development
Development teams are increasingly facing challenges regarding predictability and maintainability when requirements and context are stored exclusively in short-lived chat threads. To address this i...
agentic-software-engineering.readthedocs.io
July 20, 2026 at 5:56 AM
python4data.science
We are switching from pre-commit to prek. Whilst prek is also faster, the decisive factors were the security-related features: pinned SHA revisions and cooldown: www.python4data.science/en/latest/pr...
#Python #DX #DevOps #DevSecOps #Security
prek
prek is a variant of pre-commit, written in Rust, for managing and maintaining multilingual commit hooks. An essential task is to make the same scripts available to the entire development team. pre...
www.python4data.science
July 15, 2026 at 6:04 AM
python4data.science
We have added the `errors` argument to the instructions for reading and writing files, so that files with encoding issues can be read more effectively. And, fortunately, UTF-8 becomes the default in Python 3.15: python-basics-tutorial.readthedocs.io/en/latest/sa...
#Python
Files and Directories
pathlib implements path operations using pathlib.PurePath and pathlib.Path objects. The os and os.path modules, on the other hand, provide functions that work at a low level with str and bytes, whi...
python-basics-tutorial.readthedocs.io
July 13, 2026 at 5:47 AM
python4data.science
We have released a new version. As well as the usual software updates, it includes agent-based code generation and securing the software supply chain: www.python4data.science/en/26.1.0/
#Python #DataScience #AgenticCoding #Claude #SoftwareEngineering #SoftwareSupplyChain #SupplyChain
Python for Data Science
This is a tutorial on Data Science with Python. This immediately raises the question: What is Data Science? The term has become ubiquitous, but there is no single definition. Some even consider the...
www.python4data.science
July 8, 2026 at 12:15 PM
Reposted by Python4DataScience
veit-schiele.de
#TIL that there is also a pre-commit check for uv audit: www.python4data.science/en/latest/pr...
#Python #CVE #Vulnerability #Git #ITSecurity #SupplyChain
Security
In previous chapters, we have already provided some tips designed to help ensure safer operation. Here, we would like to summarise and expand on the individual elements once again. We will be using...
www.python4data.science
June 25, 2026 at 5:49 AM
Reposted by Python4DataScience
veit-schiele.de
Guardrails enable you to defend against prompt injection or jailbreak attacks by your coding agents. In our tutorial, we show you the options available to you for this purpose: agentic-software-engineering.readthedocs.io/en/latest/se...
#PromptInjection #Jailbreak #AI #Agentic #Coding #ITSecurity
Guardrails
The term AI guardrails first appeared in 2023, when ChatGPT launched its consumer service. The initial versions contained numerous security and privacy vulnerabilities, which ultimately prompted ma...
agentic-software-engineering.readthedocs.io
June 23, 2026 at 12:00 PM
Reposted by Python4DataScience
veit-schiele.de
We have added a section on
@tiangolo.com’s Library Skills to our agent-based software engineering tutorial: agentic-software-engineering.readthedocs.io/en/latest/sh...
#AI #Agentic #Coding #Skills
Skills
As coding agents evolve from simple chat interfaces towards autonomous task execution, context engineering has become a critical challenge. Agent Skills provide an open standard for modularising co...
agentic-software-engineering.readthedocs.io
June 22, 2026 at 5:58 AM
python4data.science
We have expanded our tutorials to include answers to the question of how coding agents can assist data scientists with programming:
• python-basics-tutorial.readthedocs.io/de/latest/li...
• github.com/veit/python-...
#Python #DataScience #CodingAgents #Claude #Cursor
Note We also use static code analysis and linting for the generated code in agent-based software development: AGENTS.md # Code quality and linting - Use ruff, ty, prek, wily for code quality and linting. - Run appropriate tooling after making changes to your code to ensure it meets quality standards. See also: Code quality and linting
June 11, 2026 at 11:58 AM
python4data.science
With uv≥0.11.19, you can check dependencies in your project for known vulnerabilities:
• python-basics-tutorial.readthedocs.io/en/latest/li...
• www.python4data.science/en/latest/pr...
#Python #uv #Vulnerability #ITSecurity
Security
In previous chapters, we have already provided some tips designed to help ensure safer operation. Here, we would like to summarise and expand on the individual elements once again. We will be using...
www.python4data.science
June 10, 2026 at 5:48 AM
Reposted by Python4DataScience
veit-schiele.de
If uv is not available in a Python environment, you can still specify the versions using pylock.toml. We have described how to do this here: python-basics-tutorial.readthedocs.io/en/latest/li...
#Python #DevSecOps #ITSecurity
Adding more Python libraries
Although Python’ „Batteries included“ philosophy means that you can already do a lot with the default installation of Python, there will inevitably come a situation where you need functionality tha...
python-basics-tutorial.readthedocs.io
June 1, 2026 at 5:42 AM
python4data.science
Thanks to Ruff, our Python4DataScience tutorial is now much more consistent: www.python4data.science/en/latest/
#Python
Python for Data Science
This is a tutorial on Data Science with Python. This immediately raises the question: What is Data Science? The term has become ubiquitous, but there is no single definition. Some even consider the...
www.python4data.science
May 22, 2026 at 5:58 AM