Skip to content

Conversation

@pk-vungle
Copy link

Hi,

I have difficulties running pip install on confluent-kafka. I would like to have the packages installed for all.

running build_ext
      building 'confluent_kafka.cimpl' extension
      creating build/temp.linux-x86_64-cpython-313/tmp/pip-install-20uri909/confluent-kafka_a0482ceaaf6647da9ca343a73d69795b/src/confluent_kafka/src
      gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I/opt/venv/include -I/usr/local/include/python3.13 -c /tmp/pip-install-20uri909/confluent-kafka_a0482ceaaf6647da9ca343a73d69795b/src/confluent_kafka/src/Admin.c -o build/temp.linux-x86_64-cpython-313/tmp/pip-install-20uri909/confluent-kafka_a0482ceaaf6647da9ca343a73d69795b/src/confluent_kafka/src/Admin.o
      error: command 'gcc' failed: No such file or directory
      [end of output]
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for confluent-kafka
Successfully built pymongo pytest-dependency jsonpath-rw
Failed to build confluent-kafka
[notice] A new release of pip is available: 25.2 -> 25.3
[notice] To update, run: pip install --upgrade pip
error: failed-wheel-build-for-install
× Failed to build installable wheels for some pyproject.toml based projects
╰─> confluent-kafka

Added gcc, g++, make, and python3-dev to the builder stage

Added gcc, g++, make, and python3-dev to the builder stage.
@cyberw
Copy link
Collaborator

cyberw commented Jan 15, 2026

Hmm... I think there was a reason I didnt add the build dependencies for all architectures, but maybe I was just over-optimizing.

But either way, how does this help you? The builder image is never published... Probably a better solution is to just install the build dependencies again in your own Dockerfile.

@pk-vungle
Copy link
Author

pk-vungle commented Jan 16, 2026

Hi, thanks for the review!

I understand your point about the builder image not being published, but this change helps in my specific workflow where I'm building multi-arch Docker images that need to install confluent-kafka during locust runtime via pip

The issue I'm facing is that when building for architectures like arm64 or others, the build dependencies for confluent-kafka need to be installed in the builder stage. Without these dependencies in the base builder image, the confluent-kafka package installation fails during my Docker build because it needs to compile C extensions.

@cyberw
Copy link
Collaborator

cyberw commented Jan 16, 2026

I understand your point about the builder image not being published, but this change helps in my specific workflow where I'm building multi-arch Docker images that need to install confluent-kafka during locust runtime via pip

That's what I don't get: Your suggested change would have no effect at all on the published image, as far as I can tell.

They would have to be installed in the runtime image (which would be adding a lot of weight to it, so that's probably not a good idea)

@pk-vungle
Copy link
Author

I managed to install kafka without this change.

@pk-vungle pk-vungle closed this Jan 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants