Implementing PostgreSQL support in Porter

[robinbraemer]

TL;DR:

Porter currently only supports MongoDB, but we're implementing direct PostgreSQL support due to limitations with MongoDB-compatible layers like FerretDB. This change will offer more deployment options, better licensing terms, and wider compatibility with other databases. We're actively working on this in our fork, aiming to merge it back into the main Porter repository. This will increase Porter's adoption, reduce vendor lock-in, and provide more performance and scalability options. We're seeking collaboration from the Porter community on this implementation.

Background

Currently, Porter strictly supports MongoDB as its database backend. While considering options for database expansion, I extensively tested FerretDB as a potential MongoDB-compatible layer for PostgreSQL. However, this approach proved problematic for several reasons:

1. Edge Cases: FerretDB has limitations that impact functionality, such as the rule that "keys should not contain dots". This and other edge cases make it challenging to use as a drop-in replacement for MongoDB.
2. Debugging Difficulties: I encountered a hard-to-debug "out of bounds" index issue, which raises concerns about production readiness and maintainability.
3. Deployment Complexity: Running FerretDB as a sidecar proxy next to Porter is not a clean solution and would likely result in a poor user experience compared to native PostgreSQL support.

These findings have led to the decision to implement direct PostgreSQL support in Porter, which will provide a more robust and user-friendly solution.

Rationale for PostgreSQL Support

There are several compelling reasons to expand our database support to include PostgreSQL:

1. Kubernetes Deployment Options:

   • MongoDB: The primary production-ready option for managing MongoDB in Kubernetes is the Percona Operator for MongoDB but even then immature ([FEAT]: Arm64 support percona/percona-server-mongodb-operator#1394).
   • PostgreSQL: There are more mature and widely-adopted options, such as the CloudNativePG operator, among others.
   • While Porter isn't strictly tied to Kubernetes, its dependency decisions significantly influence its adaptability and adoption supported by the availability of hardened k8s controllers

2. Wire Protocol Compatibility:
   PostgreSQL's wire protocol compatibility opens up possibilities for using other databases such as:

   • CockroachDB
   • YugabyteDB
   • Amazon Aurora PostgreSQL

3. Open Source Licensing:
   PostgreSQL's more permissive license is often preferred in enterprise environments concerned about licensing restrictions.

4. Enterprise Adoption and Ecosystem:
   PostgreSQL's widespread adoption in enterprise environments comes with a rich ecosystem of tools, extensions, and third-party support, beneficial for development and maintenance.

5. Managed Services Availability:
   There are more fully managed PostgreSQL services available across cloud providers, which can ease adoption and management for businesses.

Current Implementation and Proposed Changes

The current architecture includes interfaces like storage.InstallationProvider that could potentially support PostgreSQL. However, the lower-level storage.Store interface is tightly coupled to MongoDB-specific concepts.

Implementation Plan

I am actively working on implementing PostgreSQL support in our fork of Porter. This is not a proposal, but a notification of ongoing work with the intention of merging these changes back into the main Porter repository.

The implementation will follow these steps:

1. Implement the storage.InstallationProvider, storage.ParameterSetProvider, storage.CredentialSetProvider interfaces for PostgreSQL.
2. Refactor the use of the storage.Store and storage.Provider interfaces to be less dependent on them.

Benefits of Database Flexibility

1. Wider Adoption: Supporting PostgreSQL could make Porter more attractive to organizations that prefer or require PostgreSQL or its wire-compatible alternatives. (re: Our CNCF status is based on having adopters. Are you using Porter? #1596)
2. Reduced Lock-in: Offering multiple database options reduces vendor lock-in concerns and aligns with cloud-native principles of flexibility and choice.
3. Performance and Scalability Options: Different databases may offer varying performance and scalability characteristics, giving users more optimization possibilities for their specific use cases.

Call for Collaboration

While I am proceeding with the implementation, I welcome collaboration and input from the Porter maintainers and community. Your expertise and insights would be valuable in ensuring that this enhancement aligns well with Porter's architecture and future direction.

Next Steps for the Porter Team

1. Review the proposed changes and provide feedback on the approach.
2. Discuss any potential challenges or considerations for integrating PostgreSQL support into the main Porter codebase.

I believe this enhancement will significantly benefit Porter and its users, enhancing its appeal in various deployment scenarios and enterprise environments. I look forward to working with the team to integrate these changes and expand Porter's database flexibility.

Checklist

• An announcement of this proposal has been sent to the Porter mailing list: https://porter.sh/mailing-list
• This proposal has remained open for at least one week, to allow time for community feedback.

[kichristensen]

@robinbraemer Thank you so much for taking a stab at that. It is one of the things, that we never really got around to, but talk about once in a while.
It would be a great addition to Porter. Looking forward to see more details, and hope it will open up for other types of storage implementations, both relational and non-relational storage types 🤞

[kichristensen]

@robinbraemer Again thank you for working on this. Would it be possible for you to attend the community meeting tomorrow, Thursday, 6PM CEST? Where a couple of the maintainers will be present