elastic bean stalk failed to deploy with docker-compose, failed with error:open /var/pids/eb-docker-compose-log.pid: no such file or directory

Question

I have dockerised a .net application. The Dockerfile is inside the app folder and not in the root of repository

docker file is pasted below

FROM mcr.microsoft.com/dotnet/aspnet:5.0-buster-slim AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443

FROM mcr.microsoft.com/dotnet/sdk:5.0-buster-slim AS build
WORKDIR /src
COPY ["app/app.csproj", "app/"]
RUN dotnet restore "app/app.csproj"
COPY . .
WORKDIR "/src/app"
RUN dotnet build "app.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "app.csproj" -c Release -o /app/publish

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "app.dll"]

to make Elastic Beanstalk read the dockerfile I have put the following docker-compose.yml in the root of repository

version: '3.4'

services:
  app:
    image: app
    build:
      context: .
      dockerfile: app/Dockerfile

Beanstalk is building the dockerimage successfully but after that it is removed. The eb-engine.log is showing the Error

[ERROR] update processes [cfn-hup eb-docker-compose-events docker eb-docker-events healthd eb-docker-compose-log] pid symlinks failed with error Read pid source file /var/pids/eb-docker-compose-log.pid failed with error:open /var/pids/eb-docker-compose-log.pid: no such file or directory

Answer 1

I had the same

[ERROR] update processes [cfn-hup eb-docker-compose-events docker eb-docker-events healthd eb-docker-compose-log] pid symlinks failed with error Read pid source file /var/pids/eb-docker-compose-log.pid failed with error:open /var/pids/eb-docker-compose-log.pid: no such file or directory

error happening for my Docker Compose app (not using .net) when trying to deploy it to my Elastic Beanstalk environment.

I found a way to fix this which is probably specific to me, but I hope it can help you solve your problem (as I'm sure your solution might be similar, or potentially completely different). I already answered this (with slight tweaks) on this similar SO question, but posting it here in hopes it helps you and future devs facing a similar problem.

---

What caused the error for me:

This ...eb-docker-compose-log.pid: no such file... error was a false error triggered by a separate issue with my application code not finding the environment variables setup in my Elastic Beanstalk environment. See below for how I found the problem, and what I did to fix it.

---

How I found my real problem:

I downloaded the Full Logs and stumbled across a different error (which wasn't shown in the logs for the Last 100 lines):

1. go to your EB environment
2. click Logs on the left nav
3. click the Request Logs dropdown button (at the top right)
4. click Full Logs
5. click the Download link once the full logs are ready

Inside of the logs, I found the real problem in the eb-docker/containers/eb-current-app/eb-stdouterr.log file, the issue being that my application code wasn't able to find the environment variables that I setup in my Elastic Beanstalk Software configuration.

In case you're curious what my error said:

panic: required key ONE_OF_MY_ENV_KEYS missing value

(I also had a couple other errors in this log that I fixed first, but fixing the error shown above is what ended up solving the ...eb-docker-compose-log.pid: no such file... problem for me).

---

How I fixed this error:

It turns out that if you use docker-compose.yml, while setting up your environment variables in your Elastic Beanstalk Software configuration, you have to make sure you use the .env file that Elastic Beanstalk creates for you; otherwise (from my own testing), EB only see's/uses the environment variable keys/values you specify in your own .env file or environment: list you can specify in docker-compose.yml.

NOTE: see the Elastic Beanstalk "Environment properties and Environment Variables" and "Referencing environment variables in containers" sections in the docs here, in particular this bit:

"Elastic Beanstalk generates a Docker Compose environment file called .env in the root directory of your application project. This file stores the environment variables you configured for Elastic Beanstalk.

Note If you include a .env file in your application bundle, Elastic Beanstalk will not generate an .env file."

I solved my problem by updating my docker-compose.yml file to point to the supposed .env file that EB would create for me (by adding env_file: .env to my services that needed it), i.e.:

version: "3"
services:
  my_service1:
    # ...
    env_file: .env
  my_service2:
    # ...
    env_file: .env