Servers and more #

We introduced Databases in Step 4; now we're going to take the scrapers we've built, and add them to an in-production Django server application!

MVC - Model View Control #

MVC is a way to divide the components of a full-stack website into smaller parts; more technically known as a software design pattern.

Models define the data that forms the software's content. Blog posts, images, etc; anything and everything that sits in a database is defined by the models.

Views control the presentation of software's content; when a web-page is visited, views determine how the content is presented.

Controllers do the heavy lifting between models and views, by retrieving the data and responsing to user input.

How does Django differ? #

The server system that we focus on in Rebel Coding is Django with puts a bit of a twist on the ol' MVC model; namely by switching out Views for Templates and referring to Controllers and Views ...

If it sounds confusing, you won't be the first to be befuddled; and it is easily comprehended once you start working with the toolset.

As a matter of fact, we don't even bother with the Template part in the remainder of this course, as we pass that task off to a JavaScript client instead.

Why Django? #

Because Django pre-packages as administrative interface, pure, plain and simple.

Models.py #

class Post(models.Model):
    title = models.CharField(max_length=255)
    pub_date = models.DateTimeField(auto_now_add=False)
    content = models.TextField()
    slug = models.SlugField(unique=True, max_length=255)

    published = models.BooleanField(default=False)
#    category = models.ForeignKey(Category, blank=True, null=True, on_delete="PROTECT")
#    tags = models.ManyToManyField(Tag, blank=True)

    class Meta:
        ordering = ['-pub_date']

    def __unicode__(self):
        return u'%s' % self.title

    def __str__(self):
        return u'%s' % self.title

Views.py #

@permission_classes([])
@authentication_classes([])
class VideosAPIView(generics.ListAPIView):
    serializer_class = VideoSerializer

    def get_queryset(self):
        queryset = Video.objects.all()
        lang = self.request.query_params.get('q', None) 
        if lang == None:
            # lang = 0
            queryset = Video.objects.filter(published=True).all()
        else:
            queryset = Video.objects.filter(published=True, target_language=lang).all()
        return queryset

URLS.py #

urlpatterns = [
     url(r'^api/searchvids$', SearchVideosAPIView.as_view(), name="search-vids"),      
     url(r'^api/imenu$', SocialAPIView.as_view(), name="social"),   
     url(r'^api/iposts$', PostsAPIView.as_view(), name="posts"),
     url(r'^api/ivids$', VideosAPIView.as_view(), name="videos"),
     url(r'^api/ivid/[-@\w]+/$', VideoAPIView.as_view(), name="video"),
     url(r'^api/iverse$', VerseAPIView.as_view(), name="verse"),     
     url(r'^api/ividask$', RequestAPIView.as_view(), name="videoask"),
     url(r'^api/songsubmit$', UserAvatarUpload.as_view(), name="songsubmit"),
     url(r'^api/getidiokers$', GetIdiokersAPIView.as_view(), name="idiokers"),
     url(r'^api/getgenres$', GetGenreAPIView.as_view(), name="musicgenres"),
]

APIS & the Django REST Framework #

As mentioned earlier, we aren't bothering with Django Templates, rather all of our data will be consumed by a JavaScript client created using the VueJS framework.

Our Django server's duty is to serve API endpoints, and we do this using the Django REST Framework .

REST stands for Representation State Transfer ; and it is a style of software architecture to define how data is transferred between web services.

Other architectures include SOAP and GraphQL .

Serializers.py #

RESTful frameworks operate by serializing, or conforming, data into a known structure and transferring this known structure between web services. And to do that we use what is called a serializer.

Just as our models define how the data is stored in our databases, serializers define the forme and shape of data that is transferred via API calls to our RESTful API endpoints.

class PostSerializer(serializers.ModelSerializer):
    class Meta:
        model = Post
        fields = '__all__'
#        fields = ('id', 'name',)

Clone SF Project #

git clone
git clone

Writing your first web-scraper #

Attach Scraper to ORM #

Learn pupa etc …

pupa init NewCity

Your First Pull Request #

Fork the project

Exploring the SF Project #