In recent years, deep learning algorithms have grown in prominence in the field of video object detection. However, in existing systems, Sobel, canny, and wavelet filter-based approaches are employed for edge recognition, but they are unable to identify edges that are blurred or occlusion. Furthermore, the inclusion of undesired data, as well as a large amount of spatiotemporal data in the video, makes video object recognition difficult. Hence, in this research, a novel Gradient Convolution Based Edge Detection has been proposed, in which the pixel connectivity map has been derived in central, radial, and angular directions to capture the rich information on the blurred or occlusion edges. Hence identified the edges even at blurred or occlusion. Moreover, the existing models like model YOLO and Faster RCNN suffered a trade-off between accuracy and speed by extracting and learning the unrelated features from images without knowing their importance level, which provokes time complexity, thus reducing the speed. To solve this, a novel multi-level attention-based RPN has been proposed, in which three attention networks are integrated with RPN to extract all the features and effectively detect the object from the video. As a result, the proposed model effectively detects the objects on the video frames by efficiently extracting and learning the features, and increasing learning speed while reducing time complexity.