Generalizing the operation of robots in dynamical environments regardless of the task complexity is one of the ultimate goals of robotics researchers. Learning from demonstration approaches supported by transfer learning and user feedback offer a remarkable solution to achieve generalization. The main idea behind such approaches is teaching robots new skills with human instructors and training parametric models with data from demonstrations to achieve and update the desired skills under changing conditions. Herein, the applications of skill transfer with reinforcement learning algorithms and the effect of user experience (UX) on learning from demonstration approaches are reviewed. This review outlines the importance of considering and evaluating UX during human–robot interaction and, especially, robot teaching. A detailed view on the relations between robot learning and UX is provided and approaches for future improvements are derived. Finally, adaptive autonomy sharing between the robot and the user during teaching is presented as a promising approach to enhance the interaction by exploiting user feedback. In the long run, interactive and user-centered human–robot skill transfer is expected to reduce cognitive and physical load of the user. Discussion on future research questions aiming to improve learning process and semiautonomous behavior concludes the review.