The Department of Prime Minister and Cabinet has compiled a large collection of speeches, interviews, press releases etc by Australian prime ministers and made them available through the PM Transcripts site. You can search through the transcripts, and download the underlying XML files using a simple API. Back in 2016 (and again in 2019), I downloaded all the XML-formatted transcript files and shared them through a GitHub repository to make it easier for researchers to use the collection as a whole. Many more transcripts have been added in the years since, so an update was long overdue! Last week I reharvested the complete collection, which now comprises 28,560 transcripts. The only change to the harvesting process was that the file that lists all of the transcript file names is now generated by a batch process and has to be downloaded manually. Once that was saved, I could just plug it in to the old code. As previously, I created a CSV index to the transcripts, and did a little analysis and visualisation. Here, for example, is the number of transcripts by year and prime minister – you can see a lot of Scomo and Albo has been added since my original harvest. I also aggregated the transcript files by prime minister – creating a single text file for each PM, as well as a zip file containing all the XML transcripts. That should make it easier to load up all the words of a particular PM for analysis. I completely overhauled the PM Transcripts section of the GLAM Workbench in the process, updating it to the latest format. There you’ll find all the notebooks I used to process the data, as well as details of the complete dataset. The transcripts are made available by the Commonwealth under a CC-BY licence.

Use this package to identify indicators of transparency within the published literature. It can identify and extract text related to indicators of transparency from specifically formatted TXT files and from PMC XML files (i.e. XML files downloaded from the PubMed Central). It builds on the original 'rtransparent' tool of Serghiou et al. (2021) &lt;<a href="https://doi.org/10.1371%2Fjournal.pbio.3001107" target="_top">doi:10.1371/journal.pbio.3001107</a>&gt;.

Use this package to identify indicators of transparency within the published literature. It can identify and extract text related to indicators of transparency from specifically formatted TXT files and from PMC XML files (i.e. XML files downloaded from the PubMed Central). It builds on the original 'rtransparent' tool of Serghiou et al. (2021) &lt;<a href="https://doi.org/10.1371%2Fjournal.pbio.3001107" target="_top">doi:10.1371/journal.pbio.3001107</a>&gt;.

Search and download structured M&A agreements from SEC EDGAR. Tag clauses, extract terms, and export CSVs.

If an external system sends messages to a queue that we're going to read, those messages will probably be sent as JSON or XML. We added a message formatted as JSON

If an external system sends messages to a queue that we're going to read, those messages will probably be sent as JSON or XML. We added a message formatted as JSON