New modules: Llamacpp-python/run and huggingface/download for allowing to run simple text workloads with local LLMs

[toniher]

This pull request, contributed jointly with @lucacozzuto , provides a simple workload for running text inference tasks using llamacpp-python against local LLMs.
This effort was worked on during the nf-core Hackathon in March 2026.

PR checklist

Closes #XXX

• This comment contains a description of changes (with reason).
• If you've fixed a bug or added code that should be tested, add tests!
• If you've added a new tool - have you followed the module conventions in the contribution docs
• If necessary, include test data in your PR.
• Remove all TODO statements.
• Broadcast software version numbers to topic: versions - See version_topics
• Follow the naming conventions.
• Follow the parameters requirements.
• Follow the input/output options guidelines.
• Add a resource label
• Use BioConda and BioContainers if possible to fulfil software requirements.
• Ensure that the test works with either Docker / Singularity. Conda CI tests can be quite flaky:
  • For modules:
    • nf-core modules test <MODULE> --profile docker
    • nf-core modules test <MODULE> --profile singularity
    • nf-core modules test <MODULE> --profile conda
  • For subworkflows:
    • nf-core subworkflows test <SUBWORKFLOW> --profile docker
    • nf-core subworkflows test <SUBWORKFLOW> --profile singularity
    • nf-core subworkflows test <SUBWORKFLOW> --profile conda

[famosab]

Thank you for your contribution to nf-core! We really appreciate it. I added a few comments to your PR.

We usually recommend to have one module per PR. That makes the review process easier and its more likely that someone will review your PR. You can keep that in mind for the next PRs.

[toniher]

Thank you for your contribution to nf-core! We really appreciate it. I added a few comments to your PR.

We usually recommend to have one module per PR. That makes the review process easier and its more likely that someone will review your PR. You can keep that in mind for the next PRs.

Hi @famosab . Thanks for the feedback and I will go through your comments! I was told about the module submission, but since you need the output of one of the processes for the other, I thought it would help potential users once it could become eventually accepted. But, certainly, is more work for everyone. Sorry about this and I will avoid it in future PRs.

[famosab]

I think from my side it looks good, but now I spend so much time back-and-forth with you and this module is a bit unusual :) so I would like to have another set of eyes on this PR! I will ask for another review :)

[pinin4fjords]

I really think we should avoid using module binaries until we're satisfied they don't limit module portability. Currently Wave is required for Cloud scenarios without a shared file system.

Templates are the more portable, if less pretty, approach.

[pinin4fjords]

I think there's a few things to resolve here since we're colouring outside the lines a bit.

[pinin4fjords]

Are you sure you need this? A little bit of AI chat suggests that:

llama-cpp-python compiled with CUDA support does runtime GPU detection and falls back to CPU when no GPU is present

... which, if correct, would mean you could just supply the same container in either case and avoid the complexity.

You might be able to do a multi-stage thing like this to bring the container size down:

  FROM nvidia/cuda:12.4.1-devel-ubuntu22.04 AS builder                                                                         
   
  RUN apt-get update && apt-get install -y python3 python3-pip python3-dev                                                     
  RUN pip3 install --prefix=/install llama-cpp-python==0.3.16 \
      --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu124                                                   
                  
  FROM nvidia/cuda:12.4.1-runtime-ubuntu22.04                                                                                  
                  
  RUN apt-get update && apt-get install -y --no-install-recommends python3 python3-pip \                                       
      && rm -rf /var/lib/apt/lists/*
                                                                                                                               
  COPY --from=builder /install /usr/local

... so it's less overhead for the non-GPU case (untested).

[toniher]

I just tried again running Docker in my laptop (without GPU) using quay.io/nf-core/llama-cpp-python:0.1.9 and I cannot make it work.

│     RuntimeError: Failed to load shared library '/usr/local/lib/python3.10/dist-packages/llama_cpp/lib/libllama.so': libcuda.so.1: cannot open shared object file: No    │
│ such file or directory

So, it looks that after building the Docker image with GPU support, you cannot run it without any GPU device. It's particularly annoying because there are different GPU types and CUDA versions. :(

[pinin4fjords]

No singularity?

[toniher]

Related discussed here: #11053 (comment)

[toniher]

@pinin4fjords Added Singularity equivalent 9069b44 - What else would you miss so far considering the present restrictions?

[toniher]

Hi, any feedback? So far, the only weak point I see is the way to handle container GPU-enabled scenarios. Please let me know!

[mashehu]

since this tool is on conda-forge, you don't need to add a dockerfile, just use seqera containers https://nf-co.re/docs/developing/containers/seqera-containers

[toniher]

I am actually using seqera containers for no gpu situations.

https://seqera.io/containers/?packages=conda-forge::llama-cpp-python=0.3.16

I understand I could approach it similarly as here:

modules/modules/nf-core/multiqc/meta.yml

Line 110 in dd6396b

containers:

Until there is any better solution, I could enable when it is, at the same time containers, accelerator (and amd64?) to use quay.io/nf-core/llama-cpp-python:0.1.9 and for the rest of situations to do as with the multiqc module.
Otherwise, I could simply remove the gpu container until there must be a better idea. It must be said that the speed gain is huge with gpu...
Any feedback is appreciated!

[toniher]

I really think we should avoid using module binaries until we're satisfied they don't limit module portability. Currently Wave is required for Cloud scenarios without a shared file system.

Templates are the more portable, if less pretty, approach.

Any module example you could suggest as a model suitable for this case?

[pinin4fjords]

I really think we should avoid using module binaries until we're satisfied they don't limit module portability. Currently Wave is required for Cloud scenarios without a shared file system.
Templates are the more portable, if less pretty, approach.

Any module example you could suggest as a model suitable for this case?

Just look for the template keyword in the modules repo. But here are a couple of examples:

• https://github.com/nf-core/modules/tree/master/modules/nf-core/ea-utils/gtf2bed
• https://github.com/nf-core/modules/blob/master/modules/nf-core/custom/catadditionalfasta/main.nf

Note that these are not quite script files, and you have to escape $ etc with \.

[toniher]

@pinin4fjords I tried to make it work, taking your modules as examples. It was relatively easy using:

if __name__ == "__main__":
    main("--model ${gguf_model} --messages ${prompt_file} --output ${prefix}.txt ${args}")

and a bit more code bits, but... I got another issue when running the test:

│   Caused by:                                                                              │
│     Process output of type 'eval' is only allowed with Bash process scripts -- Current    │
│ interpreter: /usr/bin/env python3

This is caused by the recent convention in the way to generate versions...

    tuple val("${task.process}"), val("llama-cpp-python"), eval("python3 -c 'import llama_cpp; print(llama_cpp.__version__)'"), topic: versions, emit: versions_llama_cpp_python

So, I guess I would need to generate versions.yml the old way back both inside the script and in the stub 😿

[pinin4fjords]

So, I guess I would need to generate versions.yml the old way back both inside the script and in the stub 😿

Yep, the eval won't work, just do exactly what the gtf2bed example I gave you does. You still output the versions.yml, but also send it to a topic. The code used by pipelines to assemble versions handles both files and strings.

[pinin4fjords]

Did a bit of AI-assisted iteration to make biocorecrg#1 - see what you think. We're still trying to figure out the standard patterns for this- see nf-core/website#4142.