Does APIG Support Custom Authentication?

Yes.

For details, see section "Custom Authorizers" in the API Gateway User Guide.