It has been three months since the first confirmed case of coronavirus disease 2019 (COVID-19) in Hong Kong, and people now have a more complete picture of the extent of the pandemic. Therefore, it is time to evaluate the impacts of COVID-19 on mental health. The current population-based study aimed to evaluate the depression and anxiety of people in Hong Kong during the COVID-19 pandemic. Respondents were randomly recruited and asked to complete a structured questionnaire, including the patient health questionnaire-9 (PHQ-9), the generalized anxiety disorder-7 (GAD-7), the global rating of change scale and items related to COVID-19. Of the 500 respondents included in the study, 19% had depression (PHQ-9 score ≥ 10) and 14% had anxiety (GAD score ≥ 10). In addition, 25.4% reported that their mental health had deteriorated since the pandemic. Multiple logistic regression analysis found that not experiencing the SARS outbreak in 2003, being worried about being infected by COVID-19, being bothered by having not enough surgical masks and being bothered by not being able to work from home were associated with a poorer mental health status. Psychological support, such as brief, home-based psychological interventions, should be provided to citizens during the pandemic.