3

Once again I shall require help from Stack Overflow :).

We have a fresh public access endpoint EKS Cluster, an app inside the nodes that return something from the RDS. The VPC of the cluster is VPC peering with the private VPC that holds the RDS. We also have Accepter DNS resolution enabled. The Accepter is the RDS VPC.

When SSH-ing into my worker nodes, and we telnet the RDS, it resolves it. Initially, the Connection String was establish with the Endpoint. It didn't reach the database. I changed it to the IP of the RDS and it worked.

When doing with the DNS names, it takes:

1) lots of time to load,

2)

"Unable to retrieve Error: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached."

Therefore I was wondering if any of you faced this issue and how you solved it? There seems to be a lot of fun regarding DNS resolution with EKS and I'm not exactly sure why the instance can resolve but not the pod.

Thank you for your help!

PjoterS
  • 12,841
  • 1
  • 22
  • 54
shrimpy
  • 653
  • 3
  • 11
  • 27
  • Edit: We added 53 outbound in NaCL and now we can't even resolve the ip of the cluster anymore – shrimpy Apr 30 '19 at 11:02
  • Did you add port 53 UDP on the inbound? I had a deployment with a target in my own private subnet and not enabling the incoming DNS response seemed to be the culprit. I'm not sure how exactly RDS's endpoint URL behaves in the context of a private subnet, but perhaps you can consider manually adding a DNS entry in a private hosted zone. We use that for our workloads and it's working just fine. – Frank Yucheng Gu Apr 30 '19 at 15:35

1 Answers1

7

Okay so we found the answer! It was SO LONG to find it, so i'm gonna save you that trouble if you happen to have the same problem/configuration than us.

  1. You need port 53 outbound in NaCL and SG. That's the way kubernetes checks DNS. (DNS problem on AWS EKS when running in private subnets)
  2. In the connection string, Data source, we previously had "Data Source=DNSName;etc". We changed it to "Data source=tcp:DNSName".

That was it

2 days for that. :D

EDIT: I might add I faced the same problem in another environment/aws account (53 was the answer but slightly differently): Pods in EKS: can't resolve DNS (but can ping IP)

shrimpy
  • 653
  • 3
  • 11
  • 27