Python4DataScience
@python4data.science
Teaching materials for the cusy training courses on a Python-based data science workflow: https://cusy.io/seminars
Having first taken a closer look at CSVW for table dialects and schemas, we have now examined data resource and data package formats in detail, so that we can publish and cite data more effectively: www.python4data.science/en/latest/pr...
#Python #DataScience
#Python #DataScience
Cite data
Similar to the way in which researchers routinely provide bibliographical references to other academic sources, data should also be cited. Although data is frequently shared, it is not always cited...
www.python4data.science
September 23, 2026 at 12:04 PM
Having first taken a closer look at CSVW for table dialects and schemas, we have now examined data resource and data package formats in detail, so that we can publish and cite data more effectively: www.python4data.science/en/latest/pr...
#Python #DataScience
#Python #DataScience
CSVW is a metadata standard for CSV files. If it becomes widely adopted, it will make processing CSV files so much better: www.python4data.science/en/latest/da...
#Python #DataScience
#Python #DataScience
CSV
Overview:,,, Data structure support,--, CSV is used to store tabular data, but unlike other serialisation formats reviewed here, it’s not suitable for (nested) objects.,, Standardisation,--, CSV is...
www.python4data.science
September 21, 2026 at 12:13 PM
CSVW is a metadata standard for CSV files. If it becomes widely adopted, it will make processing CSV files so much better: www.python4data.science/en/latest/da...
#Python #DataScience
#Python #DataScience
We have significantly revised the section on data validation and cleansing and expanded it to include data quality criteria and processes: www.python4data.science/en/latest/cl...
#Python #DataScience
#Python #DataScience
Cleaning and validating data
‘Garbage in, garbage out’ is a stark reminder that it is almost impossible to draw meaningful insights from poor-quality data. Whilst there are highly regulated, safety-critical sectors in which da...
www.python4data.science
September 18, 2026 at 1:34 PM
We have significantly revised the section on data validation and cleansing and expanded it to include data quality criteria and processes: www.python4data.science/en/latest/cl...
#Python #DataScience
#Python #DataScience
We have updated the section on text comparison – featuring difflib, TheFuzz and textacy: www.python4data.science/en/latest/cl... #Python
String comparisons
difflib: You can compare strings using the get_close_matches option of Python’s built-in difflib library. If you want to specify the minimum match score required for a suggestion to be displayed, y...
www.python4data.science
September 3, 2026 at 6:03 AM
We have updated the section on text comparison – featuring difflib, TheFuzz and textacy: www.python4data.science/en/latest/cl... #Python
At first, it can sometimes be difficult to imagine a practical use for decorators. We will therefore now present a range of different examples: python-basics-tutorial.readthedocs.io/en/latest/fu...
#Python #Decorators
#Python #Decorators
Decorators
Functions can also be passed as arguments to other functions, and the results of other functions can be returned. For example, it is possible to write a Python function that takes another function ...
python-basics-tutorial.readthedocs.io
August 18, 2026 at 6:10 AM
At first, it can sometimes be difficult to imagine a practical use for decorators. We will therefore now present a range of different examples: python-basics-tutorial.readthedocs.io/en/latest/fu...
#Python #Decorators
#Python #Decorators
Narwhals provides a lightweight and extensible compatibility layer between various DataFrame libraries: www.python4data.science/en/latest/pe...
#Python #pandas #DataFrame
#Python #pandas #DataFrame
Parallelise pandas
In Enhancing performance, some possibilities are described for improving the performance of pandas. However, there are also special libraries that can parallelise the processing of data frames. cuD...
www.python4data.science
August 11, 2026 at 11:58 AM
Narwhals provides a lightweight and extensible compatibility layer between various DataFrame libraries: www.python4data.science/en/latest/pe...
#Python #pandas #DataFrame
#Python #pandas #DataFrame
We have expanded the section on mocks for testing and added spies: python-basics-tutorial.readthedocs.io/en/latest/te...
#Python #Testing
#Python #Testing
Mock
In this chapter, we will test the CLI. To do this, we will use the mock library, which has been included as part of the standard Python library since Python 3.3 under the name unittest.mock. Object...
python-basics-tutorial.readthedocs.io
August 5, 2026 at 12:46 PM
We have expanded the section on mocks for testing and added spies: python-basics-tutorial.readthedocs.io/en/latest/te...
#Python #Testing
#Python #Testing
🧵 1/2 Attacks on the software supply chain are increasing rapidly. We have therefore updated our guide on how to make your projects more secure, adding answers to the following questions:
Security
As of July 2026, PyPI hosts over 750,000 packages, and this number is growing daily. An average Python project includes dozens of additional dependencies – packages that you have never explicitly c...
www.python4data.science
August 3, 2026 at 6:13 AM
🧵 1/2 Attacks on the software supply chain are increasing rapidly. We have therefore updated our guide on how to make your projects more secure, adding answers to the following questions:
Did you know that with shot-scraper video you can explain your user interface using a storyboard: python-basics-tutorial.readthedocs.io/en/latest/do...
#Python #Docs #Video
#Python #Docs #Video
shot-scraper
shot-scraper is a tool to automate the process of updating screenshots. Installation: Use: shot-scraper can be used in several ways …for single screenshots on the command line: …or with additional ...
python-basics-tutorial.readthedocs.io
July 24, 2026 at 7:29 AM
Did you know that with shot-scraper video you can explain your user interface using a storyboard: python-basics-tutorial.readthedocs.io/en/latest/do...
#Python #Docs #Video
#Python #Docs #Video
Software Bill of Materials (SBOMs) are commonly used to check software and its dependencies for vulnerabilities using CVE and OSV. We have described how to create these for Python packages here: python4data.science/en/latest/pr...
#Python #SBOM #Vulnerability
#Python #SBOM #Vulnerability
Software Bill of Materials (SBOM)
A Software Bill of Materials (SBOM) is a document used to exchange information about software and its composition. This format is primarily used in the security field to check software and its depe...
python4data.science
July 22, 2026 at 5:50 AM
Software Bill of Materials (SBOMs) are commonly used to check software and its dependencies for vulnerabilities using CVE and OSV. We have described how to create these for Python packages here: python4data.science/en/latest/pr...
#Python #SBOM #Vulnerability
#Python #SBOM #Vulnerability
Specification-Driven Development (SDD) was developed to improve the predictability and maintainability of agentic software development. In particular, we are continuing to give OpenSpec a chance: agentic-software-engineering.readthedocs.io/en/latest/pr...
#CodingAgent #SDD #DX
#CodingAgent #SDD #DX
Spec-Driven Development
Development teams are increasingly facing challenges regarding predictability and maintainability when requirements and context are stored exclusively in short-lived chat threads. To address this i...
agentic-software-engineering.readthedocs.io
July 20, 2026 at 5:56 AM
Specification-Driven Development (SDD) was developed to improve the predictability and maintainability of agentic software development. In particular, we are continuing to give OpenSpec a chance: agentic-software-engineering.readthedocs.io/en/latest/pr...
#CodingAgent #SDD #DX
#CodingAgent #SDD #DX
We are switching from pre-commit to prek. Whilst prek is also faster, the decisive factors were the security-related features: pinned SHA revisions and cooldown: www.python4data.science/en/latest/pr...
#Python #DX #DevOps #DevSecOps #Security
#Python #DX #DevOps #DevSecOps #Security
prek
prek is a variant of pre-commit, written in Rust, for managing and maintaining multilingual commit hooks. An essential task is to make the same scripts available to the entire development team. pre...
www.python4data.science
July 15, 2026 at 6:04 AM
We are switching from pre-commit to prek. Whilst prek is also faster, the decisive factors were the security-related features: pinned SHA revisions and cooldown: www.python4data.science/en/latest/pr...
#Python #DX #DevOps #DevSecOps #Security
#Python #DX #DevOps #DevSecOps #Security
We have added the `errors` argument to the instructions for reading and writing files, so that files with encoding issues can be read more effectively. And, fortunately, UTF-8 becomes the default in Python 3.15: python-basics-tutorial.readthedocs.io/en/latest/sa...
#Python
#Python
Files and Directories
pathlib implements path operations using pathlib.PurePath and pathlib.Path objects. The os and os.path modules, on the other hand, provide functions that work at a low level with str and bytes, whi...
python-basics-tutorial.readthedocs.io
July 13, 2026 at 5:47 AM
We have added the `errors` argument to the instructions for reading and writing files, so that files with encoding issues can be read more effectively. And, fortunately, UTF-8 becomes the default in Python 3.15: python-basics-tutorial.readthedocs.io/en/latest/sa...
#Python
#Python
We have released a new version. As well as the usual software updates, it includes agent-based code generation and securing the software supply chain: www.python4data.science/en/26.1.0/
#Python #DataScience #AgenticCoding #Claude #SoftwareEngineering #SoftwareSupplyChain #SupplyChain
#Python #DataScience #AgenticCoding #Claude #SoftwareEngineering #SoftwareSupplyChain #SupplyChain
Python for Data Science
This is a tutorial on Data Science with Python. This immediately raises the question: What is Data Science? The term has become ubiquitous, but there is no single definition. Some even consider the...
www.python4data.science
July 8, 2026 at 12:15 PM
We have released a new version. As well as the usual software updates, it includes agent-based code generation and securing the software supply chain: www.python4data.science/en/26.1.0/
#Python #DataScience #AgenticCoding #Claude #SoftwareEngineering #SoftwareSupplyChain #SupplyChain
#Python #DataScience #AgenticCoding #Claude #SoftwareEngineering #SoftwareSupplyChain #SupplyChain
Reposted by Python4DataScience
#TIL that there is also a pre-commit check for uv audit: www.python4data.science/en/latest/pr...
#Python #CVE #Vulnerability #Git #ITSecurity #SupplyChain
#Python #CVE #Vulnerability #Git #ITSecurity #SupplyChain
Security
In previous chapters, we have already provided some tips designed to help ensure safer operation. Here, we would like to summarise and expand on the individual elements once again. We will be using...
www.python4data.science
June 25, 2026 at 5:49 AM
#TIL that there is also a pre-commit check for uv audit: www.python4data.science/en/latest/pr...
#Python #CVE #Vulnerability #Git #ITSecurity #SupplyChain
#Python #CVE #Vulnerability #Git #ITSecurity #SupplyChain
Reposted by Python4DataScience
Guardrails enable you to defend against prompt injection or jailbreak attacks by your coding agents. In our tutorial, we show you the options available to you for this purpose: agentic-software-engineering.readthedocs.io/en/latest/se...
#PromptInjection #Jailbreak #AI #Agentic #Coding #ITSecurity
#PromptInjection #Jailbreak #AI #Agentic #Coding #ITSecurity
Guardrails
The term AI guardrails first appeared in 2023, when ChatGPT launched its consumer service. The initial versions contained numerous security and privacy vulnerabilities, which ultimately prompted ma...
agentic-software-engineering.readthedocs.io
June 23, 2026 at 12:00 PM
Guardrails enable you to defend against prompt injection or jailbreak attacks by your coding agents. In our tutorial, we show you the options available to you for this purpose: agentic-software-engineering.readthedocs.io/en/latest/se...
#PromptInjection #Jailbreak #AI #Agentic #Coding #ITSecurity
#PromptInjection #Jailbreak #AI #Agentic #Coding #ITSecurity
Reposted by Python4DataScience
We have added a section on
@tiangolo.com’s Library Skills to our agent-based software engineering tutorial: agentic-software-engineering.readthedocs.io/en/latest/sh...
#AI #Agentic #Coding #Skills
@tiangolo.com’s Library Skills to our agent-based software engineering tutorial: agentic-software-engineering.readthedocs.io/en/latest/sh...
#AI #Agentic #Coding #Skills
Skills
As coding agents evolve from simple chat interfaces towards autonomous task execution, context engineering has become a critical challenge. Agent Skills provide an open standard for modularising co...
agentic-software-engineering.readthedocs.io
June 22, 2026 at 5:58 AM
We have added a section on
@tiangolo.com’s Library Skills to our agent-based software engineering tutorial: agentic-software-engineering.readthedocs.io/en/latest/sh...
#AI #Agentic #Coding #Skills
@tiangolo.com’s Library Skills to our agent-based software engineering tutorial: agentic-software-engineering.readthedocs.io/en/latest/sh...
#AI #Agentic #Coding #Skills
We have expanded our tutorials to include answers to the question of how coding agents can assist data scientists with programming:
• python-basics-tutorial.readthedocs.io/de/latest/li...
• github.com/veit/python-...
#Python #DataScience #CodingAgents #Claude #Cursor
• python-basics-tutorial.readthedocs.io/de/latest/li...
• github.com/veit/python-...
#Python #DataScience #CodingAgents #Claude #Cursor
June 11, 2026 at 11:58 AM
We have expanded our tutorials to include answers to the question of how coding agents can assist data scientists with programming:
• python-basics-tutorial.readthedocs.io/de/latest/li...
• github.com/veit/python-...
#Python #DataScience #CodingAgents #Claude #Cursor
• python-basics-tutorial.readthedocs.io/de/latest/li...
• github.com/veit/python-...
#Python #DataScience #CodingAgents #Claude #Cursor
With uv≥0.11.19, you can check dependencies in your project for known vulnerabilities:
• python-basics-tutorial.readthedocs.io/en/latest/li...
• www.python4data.science/en/latest/pr...
#Python #uv #Vulnerability #ITSecurity
• python-basics-tutorial.readthedocs.io/en/latest/li...
• www.python4data.science/en/latest/pr...
#Python #uv #Vulnerability #ITSecurity
Security
In previous chapters, we have already provided some tips designed to help ensure safer operation. Here, we would like to summarise and expand on the individual elements once again. We will be using...
www.python4data.science
June 10, 2026 at 5:48 AM
With uv≥0.11.19, you can check dependencies in your project for known vulnerabilities:
• python-basics-tutorial.readthedocs.io/en/latest/li...
• www.python4data.science/en/latest/pr...
#Python #uv #Vulnerability #ITSecurity
• python-basics-tutorial.readthedocs.io/en/latest/li...
• www.python4data.science/en/latest/pr...
#Python #uv #Vulnerability #ITSecurity
Reposted by Python4DataScience
If uv is not available in a Python environment, you can still specify the versions using pylock.toml. We have described how to do this here: python-basics-tutorial.readthedocs.io/en/latest/li...
#Python #DevSecOps #ITSecurity
#Python #DevSecOps #ITSecurity
Adding more Python libraries
Although Python’ „Batteries included“ philosophy means that you can already do a lot with the default installation of Python, there will inevitably come a situation where you need functionality tha...
python-basics-tutorial.readthedocs.io
June 1, 2026 at 5:42 AM
If uv is not available in a Python environment, you can still specify the versions using pylock.toml. We have described how to do this here: python-basics-tutorial.readthedocs.io/en/latest/li...
#Python #DevSecOps #ITSecurity
#Python #DevSecOps #ITSecurity
Thanks to Ruff, our Python4DataScience tutorial is now much more consistent: www.python4data.science/en/latest/
#Python
#Python
Python for Data Science
This is a tutorial on Data Science with Python. This immediately raises the question: What is Data Science? The term has become ubiquitous, but there is no single definition. Some even consider the...
www.python4data.science
May 22, 2026 at 5:58 AM
Thanks to Ruff, our Python4DataScience tutorial is now much more consistent: www.python4data.science/en/latest/
#Python
#Python
Reposted by Python4DataScience
Yesterday, the article by Frank Hofmann and me on open licences for documents, images, audio, video, fonts and hardware was published in Linux Magazine: www.linux-magazine.com/Issues/2026/...
#OpenSource #OpenHardware
#OpenSource #OpenHardware
Keep It Free » Linux Magazine
Linux users associate open source licenses with software, but a bevy of licenses are available for documents, images, audio/video, fonts, and hardw...
www.linux-magazine.com
May 7, 2026 at 7:00 AM
Yesterday, the article by Frank Hofmann and me on open licences for documents, images, audio, video, fonts and hardware was published in Linux Magazine: www.linux-magazine.com/Issues/2026/...
#OpenSource #OpenHardware
#OpenSource #OpenHardware
We have integrated two features from Git 2.54:
• Git history: www.python4data.science/en/latest/pr...
• Configuration-based Git hooks: www.python4data.science/en/latest/pr...
#Git
• Git history: www.python4data.science/en/latest/pr...
• Configuration-based Git hooks: www.python4data.science/en/latest/pr...
#Git
Rewriting history
There are several commands in Git for rewriting history. git rebase -i is the best known and most flexible: you can reorder, merge, edit and remove commits. However, this flexibility comes with a d...
www.python4data.science
April 24, 2026 at 6:03 AM
We have integrated two features from Git 2.54:
• Git history: www.python4data.science/en/latest/pr...
• Configuration-based Git hooks: www.python4data.science/en/latest/pr...
#Git
• Git history: www.python4data.science/en/latest/pr...
• Configuration-based Git hooks: www.python4data.science/en/latest/pr...
#Git
The testing section has been extensively revised:
• Test examples for cusy.tasks
• Added Hypothesis extensions
• Agile software development using test-driven development and behaviour-driven development
python-basics-tutorial.readthedocs.io/en/latest/te...
#Python #pytest #Agile #TDD #BDD
• Test examples for cusy.tasks
• Added Hypothesis extensions
• Agile software development using test-driven development and behaviour-driven development
python-basics-tutorial.readthedocs.io/en/latest/te...
#Python #pytest #Agile #TDD #BDD
Testing
Basically, a distinction is made between static and dynamic test procedures. Static test procedures, are used to check the source code, although this is not executed. They are divided into- reviews...
python-basics-tutorial.readthedocs.io
April 23, 2026 at 6:09 AM