icecheng/freeleaps-ops
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
2,057 Commits 1 Branch 0 Tags 29 MiB
c10ff2d550
Commit Graph

86 Commits

Author SHA1 Message Date
zhenyus
b54ad6a1b0 chore(starrocks): increase fe resources to 2c4g and limits to 4c8g 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-09-10 11:10:57 +08:00
zhenyus
ba78fbc007 Add resource requests and limits to Kafka cluster configuration, and update KafkaUser to reference a new secret for password management. Also, enhance KafkaUser ACLs and MongoDB connector configuration for full document change streams. 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-09-05 09:40:49 +08:00
zhenyus
aa46147e33 Add SCRAM-SHA-512 authentication to Kafka cluster configuration in freeleaps-data-platform manifests. 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-09-02 11:47:58 +08:00
zhenyus
d90e0f2692 Remove Flink and Metabase manifests from the freeleaps-data-platform directory, including CRDs, RBAC configurations, deployments, and associated resources. This cleanup streamlines the project by eliminating unused components. 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-09-02 10:38:40 +08:00
zhenyus
604af2dffb Remove Jenkins and Kubernetes overall dashboards from the monitoring system manifests. 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-09-01 00:37:14 +08:00
zhenyus
d7c072ee6a Remove deprecated Kafka and StarRocks configurations, including README, storage classes, and Vertical Pod Autoscaler files. This cleanup prepares for a more streamlined deployment process. 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-08-21 15:52:39 +08:00
Nicolas
f6c5aa12e4 deploy metabase 2025-08-21 11:07:48 +08:00
Nicolas
c9f681e44b deploy flink 2025-08-21 10:50:07 +08:00
Nicolas
f6f464dbae rewrite readme about starrocks 2025-08-20 18:53:25 +08:00
Nicolas
44f8a10431 Deploy the StarRocks Operator and use the Operator to deploy the HA StarRocks cluster 2025-08-20 18:50:45 +08:00
Nicolas
e7ec6a4258 Installed Strimzi Kafka Operator version 0.45.0 
Three Kafka nodes + Three ZooKeeper nodes
Can tolerate 1 node failure
3 replicas distributed across different nodes
Use Azure Disk SSD SCRAM-SHA-512 + ACLs
 2025-08-20 17:44:55 +08:00
Nicolas
31f959f7a9 feat: enable log collection for prod environment 
- Enable logIngest for chat and freeleaps services in prod
- Add Loki datasource to Grafana for prod environment
- Configure Loki log retention policy (30 days)
- Enable table manager for automatic log cleanup
 2025-08-13 09:41:25 +08:00
zhenyus
7a9c695c9e ci(bump): bump reconciler image version for alpha to snapshot-9f1a2bc 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-08-04 15:59:50 +08:00
zhenyus
3988ff13a8 ci(bump): update reconciler image version for alpha to 1.0.2 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-07-31 15:30:16 +08:00
zhenyus
c2d2fa6345 fix: update Jenkins token in gitea webhook configuration 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-07-24 16:51:35 +08:00
zhenyus
dca5cffa55 fix(flink): update resource requests and limits for jobmanager and taskmanager 
- Adjusted CPU and memory requests and limits for both jobmanager and taskmanager to optimize resource allocation.
- Commented out the resourcesPreset parameter for clarity in configuration.

Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-07-04 11:03:26 +08:00
zhenyus
9c07783780 feat(kafka, pinot, star-rocks): update configurations and resource limits across multiple components 
- Updated Kafka configuration to specify Kubernetes version and API versions.
- Enabled Vertical Pod Autoscaler (VPA) for Pinot and adjusted resource limits for CPU and memory.
- Removed obsolete certificate configuration for Pinot.
- Enhanced StarRocks values.yaml with comprehensive configurations for deployment, including service specifications and resource requests/limits.
- Increased timeout settings in production values for Freeleaps to improve service resilience.

Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-06-26 23:04:03 +08:00
zhenyus
db590f3f27 refactor: update gitea-webhook-ambassador Dockerfile and configuration 
- Changed the build process to include a web UI build stage using Node.js.
- Updated Go build stage to copy web UI files to the correct location.
- Removed the main.go file as it is no longer needed.
- Added SQLite database configuration to example config.
- Updated dependencies in go.mod and go.sum, including new packages for JWT and SQLite.
- Modified .gitignore to include new database and configuration files.

Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-06-10 16:00:52 +08:00
zhenyus
c8b68afc75 feat: Update Pinot configuration and RBAC rules 
- Enhanced the Pinot Helm chart values.yaml with comprehensive configurations for controller, broker, server, minion, and zookeeper components.
- Added support for pod disruption budgets and custom resource definitions in RBAC rules.
- Introduced a new script for managing Kubernetes service port forwarding, allowing users to easily forward, stop, and list active services.
- Updated helm repository list to ensure proper access to necessary charts.

Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-05-20 16:00:32 +08:00
zhenyus
db0cd26f4b feat: update RBAC configurations for data platform and mathmast roles 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-05-12 10:56:58 +08:00
zhenyus
b7c11d2829 feat: update RBAC configurations and add Jenkinsfile for aml-services 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-05-12 09:56:54 +08:00
Joe
134caeaeb2 feat: add starrocks to repo list 2025-05-10 15:48:29 +08:00
Joe
2d7b58dad7 Merge branch 'dev_zhengyang' of ssh.dev.azure.com:v3/freeleaps/freeleaps-ops/freeleaps-ops into dev_zhengyang 2025-05-09 17:53:34 +08:00
Joe
f4df870452 fix: update freeleaps-data-platform 2025-05-09 17:52:53 +08:00
zhenyus
2f7128a51c feat: update namespaces and add RBAC roles for freeleaps data platform and monitoring systems 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-05-09 13:10:13 +08:00
Zheng Yang
51cbfbef07 Deleted kafka-monitoring-table.yaml 2025-05-09 02:16:08 +00:00
Joe
a7025081a1 feat: add freeleaps-data-platform 2025-05-09 10:14:16 +08:00
Joe
b2b1fd274f feat: add freeleaps-data-platform 2025-05-09 10:07:00 +08:00
zhenyus
15dd1fba0b fix(opentelemetry): update resource attributes in distributor and log transformation for improved metadata extraction 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-04-21 17:43:08 +08:00
zhenyus
473f5cea54 fix(opentelemetry): update log transformation to use resource attributes for application and environment 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-04-21 17:17:40 +08:00
zhenyus
c106c9a624 fix(loki): update resource_attributes to use regex for indexing labels 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-04-21 17:05:53 +08:00
zhenyus
222f5ee0fb feat(opentelemetry): add structured metadata support and update log transformation logic 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-04-21 16:16:03 +08:00
zhenyus
67c4772407 fix: update imagePullPolicy to 'Always' for chat, backend, and frontend services; change branch name from 'master' to 'main' in configmap 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-04-18 14:36:35 +08:00
zhenyus
eedb1cefc7 fix: update CA injection annotations and webhook service URLs for OpenTelemetry operator 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-04-16 05:49:54 +08:00
zhenyus
a0d88d9507 Add OpenTelemetry Collector configuration for log ingestion 
- Introduced a new OpenTelemetryCollector resource in the Helm chart.
- Configured filelog receiver to ingest logs based on specified patterns.
- Added processors for Kubernetes attributes and resource metadata.
- Set up Loki exporter for log forwarding with appropriate labels.
- Configured logging verbosity and defined log processing pipelines.

Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-04-16 05:45:40 +08:00
zhenyus
decca8e7a1 fix: update labels for Fluent Bit resources to ensure correct identification 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-04-16 00:46:39 +08:00
zhenyus
358f131809 Add Fluent Bit configuration for log collection 
- Introduced Fluent Bit resources including FluentBit, Parser, Output, FluentBitConfig, and ClusterInput.
- Configured default resource requests and limits for Fluent Bit.
- Set up JSON parser with customizable time key and format.
- Established output forwarding to Fluentd service in the logging system.
- Enabled conditional deployment based on the `fluentbit.enabled` value in Helm chart.

Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-04-16 00:18:16 +08:00
zhenyus
0de22b2623 refactor(ImageBuilder): remove logging of files in artifact directory to streamline output 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-04-14 16:40:46 +08:00
zhenyus
6ebdef8933 Update ingress hosts and issuer references for alpha environment 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-04-11 01:22:29 +08:00
zhenyus
6029462f12 ci: update Gitea webhook configuration and modify Redis URLs for alpha environment 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-03-31 15:19:33 +08:00
zhenyus
32ba41f1f4 feat: add secret key configuration for webhook authentication 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-03-31 00:53:33 +08:00
zhenyus
60817c1be4 feat: add gitea-webhook-ambassador service and migration script 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-03-30 23:37:19 +08:00
zhenyus
c66408be1c fix: update groupName in ClusterIssuer for GoDaddy webhook 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-03-27 15:28:45 +08:00
zhenyus
0cecb340da feat: add ClusterIssuer for freeleaps.com and update issuer reference in frontend values 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-03-27 15:15:02 +08:00
zhenyus
39dfcd56cd feat: add Kubernetes dashboard commands and token retrieval to authenticator script 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-03-20 10:38:01 +08:00
zhenyus
073a7e0559 fix: update load-watcher image tag to v0.0.3-modify for consistency 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-03-18 11:22:37 +08:00
zhenyus
2969cafc9d feat: add serviceMonitor configuration for content, payment, notification, central-storage, and authentication services 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-03-17 21:24:35 +08:00
zhenyus
4bb5d6ee38 feat: add Vertical Pod Autoscaler components and scripts for deployment 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-03-14 12:44:49 +08:00
zhenyus
68a818de80 feat: add Descheduler Helm chart with templates and tests for deployment and cronjob 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-03-11 22:25:53 +08:00
zhenyus
98a13e4b04 fix: add tolerations and affinity rules for devops node role in Argo CD and Jenkins, and update resource requests and limits for RabbitMQ 
Signed-off-by: zhenyus <zhenyus@mathmast.com>
 2025-03-08 21:36:53 +08:00