Three Kafka nodes + Three ZooKeeper nodes
Can tolerate 1 node failure
3 replicas distributed across different nodes
Use Azure Disk SSD SCRAM-SHA-512 + ACLs
- Adjusted CPU and memory requests and limits for both jobmanager and taskmanager to optimize resource allocation.
- Commented out the resourcesPreset parameter for clarity in configuration.
Signed-off-by: zhenyus <zhenyus@mathmast.com>
- Updated Kafka configuration to specify Kubernetes version and API versions.
- Enabled Vertical Pod Autoscaler (VPA) for Pinot and adjusted resource limits for CPU and memory.
- Removed obsolete certificate configuration for Pinot.
- Enhanced StarRocks values.yaml with comprehensive configurations for deployment, including service specifications and resource requests/limits.
- Increased timeout settings in production values for Freeleaps to improve service resilience.
Signed-off-by: zhenyus <zhenyus@mathmast.com>
- Changed the build process to include a web UI build stage using Node.js.
- Updated Go build stage to copy web UI files to the correct location.
- Removed the main.go file as it is no longer needed.
- Added SQLite database configuration to example config.
- Updated dependencies in go.mod and go.sum, including new packages for JWT and SQLite.
- Modified .gitignore to include new database and configuration files.
Signed-off-by: zhenyus <zhenyus@mathmast.com>
- Enhanced the Pinot Helm chart values.yaml with comprehensive configurations for controller, broker, server, minion, and zookeeper components.
- Added support for pod disruption budgets and custom resource definitions in RBAC rules.
- Introduced a new script for managing Kubernetes service port forwarding, allowing users to easily forward, stop, and list active services.
- Updated helm repository list to ensure proper access to necessary charts.
Signed-off-by: zhenyus <zhenyus@mathmast.com>
- Introduced a new OpenTelemetryCollector resource in the Helm chart.
- Configured filelog receiver to ingest logs based on specified patterns.
- Added processors for Kubernetes attributes and resource metadata.
- Set up Loki exporter for log forwarding with appropriate labels.
- Configured logging verbosity and defined log processing pipelines.
Signed-off-by: zhenyus <zhenyus@mathmast.com>
- Introduced Fluent Bit resources including FluentBit, Parser, Output, FluentBitConfig, and ClusterInput.
- Configured default resource requests and limits for Fluent Bit.
- Set up JSON parser with customizable time key and format.
- Established output forwarding to Fluentd service in the logging system.
- Enabled conditional deployment based on the `fluentbit.enabled` value in Helm chart.
Signed-off-by: zhenyus <zhenyus@mathmast.com>
