This article describes how you can customize the YAML file, depending on what you want to accomplish in the AtScale topology. Basically, we can split most of the services into a different host, remap ports, and utilize a load balancer.
Caution
-- default engine INSERT INTO engines( engine_id, org_id, name, protocol, host, port, properties, default_engine ) VALUES ( '2591c41d-14f9-474b-6243-1a1470f80307', 'default', 'default', 'https', 'ubuntu-atscale.atscaledomain.com', '80', '{}', true ) ON CONFLICT (engine_id) DO UPDATE SET protocol='http', host='ubuntu-atscale.atscaledomain.com', port='80'; EOM ) > /dev/null exit 0
To counter that, in the yaml file, you can add internal and external hosts, but only for the load balancer DNS mapping.
This setting will allow you to extend the AtScale internal host vs. external load balancer with separate ports in the load balancer vs internal.
Here is the working customization of AtScale with two load balancers with two ports mapped on 10500 to 80 and 10502 to 80:
installation_location: "/opt/atscale" service_account: "atscale" loadbalancer_dns_name: "205.49.117.34.bc.googleusercontent.com" tls: enabled: false certificate: "/opt/atscale/conf/server.cert" key: "/opt/atscale/conf/server.key" kerberos: enabled: false keytab: "/opt/atscale/conf/atscale.keytab" principal: "atscale/rw-ubuntu-1.us-west1-b.c.atscale-sales-demo.internal@REALM" hosts: - dnsname: rw-coordinator.us-west1-a.c.atscale-sales-demo.internal services: - coordinator override: coordinator: id: 10 engine: port: 80 schema: http hostname: 38.250.117.34.bc.googleusercontent.com modeler: port: 80 schema: http hostname: 205.49.117.34.bc.googleusercontent.com - dnsname: rw-database1.us-west1-a.c.atscale-sales-demo.internal services: - database - coordinator override: coordinator: id: 20 engine: port: 80 schema: http hostname: 38.250.117.34.bc.googleusercontent.com modeler: port: 80 schema: http hostname: 205.49.117.34.bc.googleusercontent.com - dnsname: rw-database2.us-west1-a.c.atscale-sales-demo.internal services: - database - coordinator override: coordinator: id: 30 engine: port: 80 schema: http hostname: 38.250.117.34.bc.googleusercontent.com modeler: port: 80 schema: http hostname: 205.49.117.34.bc.googleusercontent.com - dnsname: rw-engine-1.us-west1-a.c.atscale-sales-demo.internal services: - engine - modeler override: engine: port: 80 schema: http hostname: 38.250.117.34.bc.googleusercontent.com modeler: port: 80 schema: http hostname: 205.49.117.34.bc.googleusercontent.com - dnsname: rw-engine2.us-west1-a.c.atscale-sales-demo.internal services: - engine - modeler override: engine: port: 80 schema: http hostname: 38.250.117.34.bc.googleusercontent.com modeler: port: 80 schema: http hostname: 205.49.117.34.bc.googleusercontent.com external: engine: load_balancer_url: http://38.250.117.34.bc.googleusercontent.com apps: load_balancer_url: http://205.49.117.34.bc.googleusercontent.com sql: load_balancer_url: http://54.208.233.35.bc.googleusercontent.com:11111 mdx: load_balancer_url: http://38.250.117.34.bc.googleusercontent.com
The following is a complete customization of what you can do in atscale.yaml
engine: allow_local_license: true agent: dcs: backend: consul logbundle_location: '{{ atscale_log_location }}/bundles' ports: http: 10525 queue: backend: consul t_ip: 'true' uwsgi_processes: 2 coordinator: base_path: /atscale id: 1 ports: client: 10513 connect: 10505 election: 10506 security: key: s59446c445a6d3437553051714e5635526547566563773d3d key_cl: '`9Cfn;SD*5^Qxe^s' key_cl_digest: sO4bf5RVbSwI1XB8ieWWm0nHc9U= user: super data: message_bus: default_replication_factor: 1 search_index: partitioning: time_zone_id: system-default streams: metrics: data_connector: buffered_writer: flush: duration: 5.seconds enabled: false error_handling: enable_error_logs: true include_error_log_messages: true tolerance: all max_poll_interval: 120.seconds max_poll_records: 20 max_tasks: 1 offset_committer: type: default queries_execution_info: data_connector: buffered_writer: flush: duration: 3.seconds enabled: false error_handling: enable_error_logs: true include_error_log_messages: true tolerance: all max_poll_interval: 120.seconds max_poll_records: 10 max_tasks: 1 offset_committer: safe_aggregation: timeout: 2.hours type: safe-aggregation search_index: partitioning: Daily refresh_interval: 3.seconds replicas: 0 shards: 1 queries_part_planned: data_connector: buffered_writer: flush: duration: 3.seconds enabled: false error_handling: enable_error_logs: true include_error_log_messages: true tolerance: all max_poll_interval: 120.seconds max_poll_records: 10 max_tasks: 1 offset_committer: safe_aggregation: timeout: 2.hours type: safe-aggregation search_index: partitioning: Daily refresh_interval: 3.seconds replicas: 0 shards: 1 queries_planned: data_connector: buffered_writer: flush: duration: 3.seconds enabled: false error_handling: enable_error_logs: true include_error_log_messages: true tolerance: all max_poll_interval: 120.seconds max_poll_records: 10 max_tasks: 1 offset_committer: safe_aggregation: timeout: 2.hours type: safe-aggregation search_index: partitioning: Daily refresh_interval: 3.seconds replicas: 0 shards: 1 query_events: data_connector: buffered_writer: flush: duration: 5.seconds enabled: false error_handling: enable_error_logs: true include_error_log_messages: true tolerance: all max_poll_interval: 120.seconds max_poll_records: 20 max_tasks: 1 offset_committer: type: default data_agent: allow_local_license: false debug: dev_mode: false kerberos: false jvm: base_java_opts: -J-Xms250m -J-server gc_opts: -J-XX:+UseG1GC -J-XX:+UseAdaptiveGCBoundary memory: 2G metricsd: jolokia: host: 127.0.0.1 port: 12502 ports: http: 10502 jmx: 13502 remoting: 10507 database: database: grafana: name: grafana name: atscale schemas: andromeda: name: andromeda atscale: name: atscale datadash: name: datadash infraapi: name: infraapi orchestrator: name: orchestrator ports: external: 10518 internal: 10520 patroni_rest: 10519 postgresql: parameters: log_autovacuum_min_duration: 10000 log_checkpoints: 'off' log_connections: 'off' log_destination: stderr log_directory: '{{ atscale_log_location }}/database' log_disconnections: 'off' log_duration: 'off' log_error_verbosity: TERSE log_file_mode: 384 log_filename: postgresql-%Y-%m-%d_%H%M%S.log log_line_prefix: '%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h ' log_lock_waits: 'on' log_min_duration_statement: 2000 log_rotation_age: 1d log_rotation_size: 0 log_temp_files: -1 log_truncate_on_rotation: 'on' logging_collector: 'on' security: atscale: key: s5958527a593246735a513d3d username: atscale replication: key: s4f474654567a5244545442434f4539585a586c76 username: atscale_repl super: key: s56586869616d3951563273335355354a5555465a username: atscale_super datadash: debug: dev_mode: false jvm: base_java_opts: -J-Xms250m -J-server gc_opts: -J-XX:+UseG1GC -J-XX:+UseAdaptiveGCBoundary memory: 2G ports: http: 10501 deployment: messaging: enabled: false mode: '{{ atscale_build_profile }}' directory: ports: egress: ldap: 10524 ldap: 10514 security: admin: key: s5958527a593246735a513d3d egress: ports: stats: 10517 timeout: client: 130m server: 130m engine: allow_local_license: false debug: dev_mode: false kerberos: false jvm: base_java_opts: -J-Xms250m -J-server gc_opts: -J-XX:+UseG1GC -J-XX:+UseAdaptiveGCBoundary memory: 8G metricsd: jolokia: host: 127.0.0.1 port: 12502 ports: egress: http: 10582 http: 10502 jmx: 13502 ping: 10587 ping_rtr: 10588 remoting: 10507 gov_enforce: ports: egress: server: 10548 server: 10547 gov_rules: ports: egress: server: 10546 server: 10545 grafana: ports: egress: http: 10597 http: 10596 host: validation: enabled: true hosts: - dnsname: '### Fully qualified domain name for the host running atscale' hostname: '### Hostname of the node if it''s different from the DNS entry. (ie: what hostname -f returns)' services: - atscale - coordinator infraapi: ports: http: 10590 uwsgi_processes: 1 ingress: keepalive_timeout: 65 worker_connections: 1024 worker_processes: 1 installation_location: '### Path where atscale is installed' kerberos: enabled: false keytab: '### Path to keytab. ex: /opt/atscale/conf/atscale.keytab' krb5conf: system principal: '### Principal associated with the keytab. ex: atscaler/host.domain@REALM' validation: enabled: true kibana: ports: egress: http: 10599 http: 10598 loadbalancer_dns_name: '### DNS entry used to get to atscale' message_bus: memory: 256M metricsd: jolokia: host: 127.0.0.1 port: 12560 ports: egress: plaintext: 10578 jmx: 13560 plaintext: 10560 tls: 10561 message_exchange: cors: allow: methods: '' origin: '' management: topics: config_storage: partitions: 1 replication_factor: 1 topic: exchange-configs offset_storage: partitions: 1 replication_factor: 1 topic: exchange-offsets status_storage: partitions: 1 replication_factor: 1 topic: exchange-status memory: 256M metricsd: jolokia: host: 127.0.0.1 port: 12567 ports: egress: rest: 10577 jmx: 13567 rest: 10567 metadata_engine: allow_local_license: false debug: dev_mode: false kerberos: false jvm: base_java_opts: -J-Xms250m -J-server gc_opts: -J-XX:+UseG1GC -J-XX:+UseAdaptiveGCBoundary memory: 2G metricsd: jolokia: host: 127.0.0.1 port: 12502 ports: http: 10502 jmx: 13502 remoting: 10507 metricsd: agent: flush_interval: 10s interval: 30s logging: rotation: interval: 1d max: archives: 10 size: 20MB target: file ports: egress: server: 10529 server: 10528 statsd: 12560 modeler: app_mode: modeler flags: skip_first_time_setup_dialog: false ports: authorization: 10503 egress: authorization: 10583 http: 10580 http: 10500 orchestrator: debug: dev_mode: false jvm: base_java_opts: -J-Xms250m -J-server gc_opts: -J-XX:+UseG1GC -J-XX:+UseAdaptiveGCBoundary memory: 2G ports: http: 10540 orchestrator_ui: app_mode: orchestrator deployment: saas flags: skip_first_time_setup_dialog: false ports: authorization: 10503 egress: authorization: 10583 http: 10591 http: 10510 query_engine: allow_local_license: false debug: dev_mode: false kerberos: false jvm: base_java_opts: -J-Xms250m -J-server gc_opts: -J-XX:+UseG1GC -J-XX:+UseAdaptiveGCBoundary memory: 2G metricsd: jolokia: host: 127.0.0.1 port: 12541 ports: http: 10541 jmx: 13541 remoting: 10507 search_index: memory: 1G ports: egress: http: 10575 http: 10570 transport: 10571 service_account: '### user running the service' service_registry: addresses: dns: 0.0.0.0 grpc: 0.0.0.0 http: 0.0.0.0 https: 0.0.0.0 base_path: /atscale bind_addr: 0.0.0.0 domain: atscale log_level: INFO ports: dns: 10554 grpc: 10557 http: 10555 https: 10556 serf_lan: 10551 serf_wan: 10552 server: 10550 ui: true servicecontrol: ports: http: 10516 tls: certificate: '### Path to certificate for TLS. ex: /opt/atscale/conf/server.cert' custom_truststore: enabled: false location: '### Path to the provided truststore.' password: '### Password for the provided truststore.' database: enabled: false default: bootstrap_tls: true ca_cert: '### Path to default_tls ca_chain, on saas this is in $HOME/root.key' ca_key: '### Path to default_tls ca_chain, on saas this is in $HOME/root.key' enabled: true enabled: false generate_certs: false internal: ca_cert: '### Path to key for TLS. ex: /opt/atscale/conf/server.key' enabled: false key: '### Path to key for TLS. ex: /opt/atscale/conf/server.key' keystore: key: s6347467a63336476636d513d validation: enabled: true vault: keyfile: '{{ atscale_version_location }}/conf/vault/unseal_key' name: vault ports: api: 10558 cluster: 10559 role_name: agent_role virtualization: tmp_dir: /tmp virtualization_listener: cores: 2 driver_port: 10531 memory: 4G name: virtualization_listener port: 10511 shuffle_partitions: 100 webui_port: 10530 virtualization_supervisor: cores: 2 manager_port: 10532 memory: 4G name: virtualization_supervisor port: 10533 webui_port: 10534 virtualization_worker: controller_process_memory: 2G cores: 2 memory: 8G name: virtualization_worker port: 10535 webui_port: 10536
Was this article helpful?
0 out of 0 found this helpful
Yes
No