To extract information from free-text in clinical records due to the patient's protected health information PHI in the records pre-processing of de-identification is required. Therefore we aimed to identify PHI list and fine-tune the deep learning BERT model for developing de-identification model. The result of fine-tuning the model is strict F1 score of 0.924. Due to the convinced score the model can be used for the development of a de-identification model.