Full Code of gvolpe/advanced-http4s for AI

Repository: gvolpe/advanced-http4s
Branch: master
Commit: f6bee1e016cb
Files: 30
Total size: 36.3 KB

Directory structure:
gitextract_r37kbv80/

├── .gitignore
├── README.md
├── build.sbt
├── project/
│   ├── Dependencies.scala
│   └── build.properties
└── src/
    ├── main/
    │   └── scala/
    │       └── com/
    │           └── github/
    │               └── gvolpe/
    │                   ├── fs2/
    │                   │   ├── Counter.scala
    │                   │   ├── Fifo.scala
    │                   │   ├── Once.scala
    │                   │   ├── PubSub.scala
    │                   │   ├── Resources.scala
    │                   │   └── package.scala
    │                   └── http4s/
    │                       ├── StreamUtils.scala
    │                       ├── client/
    │                       │   ├── MultipartClient.scala
    │                       │   └── StreamClient.scala
    │                       └── server/
    │                           ├── Module.scala
    │                           ├── Server.scala
    │                           ├── endpoints/
    │                           │   ├── FileHttpEndpoint.scala
    │                           │   ├── HexNameHttpEndpoint.scala
    │                           │   ├── JsonXmlHttpEndpoint.scala
    │                           │   ├── MultipartHttpEndpoint.scala
    │                           │   ├── TimeoutHttpEndpoint.scala
    │                           │   ├── auth/
    │                           │   │   ├── AuthRepository.scala
    │                           │   │   ├── BasicAuthHttpEndpoint.scala
    │                           │   │   └── GitHubHttpEndpoint.scala
    │                           │   └── package.scala
    │                           └── service/
    │                               ├── FileService.scala
    │                               └── GitHubService.scala
    └── test/
        └── scala/
            └── com/
                └── github/
                    └── gvolpe/
                        └── http4s/
                            ├── IOAssertion.scala
                            └── server/
                                └── endpoints/
                                    ├── HexNameHttpEndpointSpec.scala
                                    └── JsonXmlHttpEndpointSpec.scala

================================================
FILE CONTENTS
================================================

================================================
FILE: .gitignore
================================================
ml
*.ipr
*.iws
*.pyc
*.tm.epoch
*.vim
*/project/boot
*/project/build/target
*/project/project.target.config-classes
*-shim.sbt
*~
.#*
.*.swp
.DS_Store
.cache
.cache
.classpath
.codefellow
.ensime*
.eprj
.history
.idea
.manager
.multi-jvm
.project
.scala_dependencies
.scalastyle
.settings
.tags
.tags_sorted_by_file
.target
.worksheet
Makefile
TAGS
lib_managed
logs
project/boot/*
project/plugins/project
src_managed
target
tm*.lck
tm*.log
tm.out
worker*.log
/bin
t
tags
tq:q
TODO


================================================
FILE: README.md
================================================
advanced-http4s
===============

Code samples of advanced features of [Http4s](http://http4s.org/) in combination with some features of [Fs2](https://functional-streams-for-scala.github.io/fs2/) not often seen.

Streaming end to end
--------------------

- **Server**: Streaming responses end to end, from the `FileService` reading all the directories in your `$HOME` directory to the `FileHttpEndpoint`.
- **StreamClient**: Parsing chunks of the response body produced by the server in a streaming fashion way.

> You'll need two sbt sessions. Run the server in one and after the client in the other to try it out.

Middleware Composition
----------------------

- **GZip**: For compressed responses. Client must set the `Accept Encoding` header to `gzip`.
- **AutoSlash**: To make endpoints work with and without the slash `/` at the end.

> Response compression is verified by `HexNameHttpEndpointSpec`. You can also try it out on Postman or similar.

- **Timeout**: Handling response timeouts with the given middleware.

> The `TimeoutHttpEndpoint` generates a response in a random time to demonstrate the use.

- **NonStreamResponse**: Using the `ChunkAggregator` middleware to wrap the streaming `FileHttpEndpoint` and remove the Chunked Transfer Encoding.

> The endpoint `/v1/nonstream/dirs?depth=3` demonstrates the use case.

Media Type negotiation
----------------------

- **XML** and **Json**: Decoding request body with either of these types for the same endpoint.

> The `JsonXmlHttpEndpoint` demonstrates this use case and it's validated in its spec.

Multipart Form Data
-------------------

- **Server**: The `MultipartHttpEndpoint` is responsible for parsing multipart data with the given multipart decoder.
- **MultipartClient**: Example uploading both text and an image.

> Similar to the streaming example, you'll need to run both Server and MultipartClient to see how it works.

*NOTE: Beware of the creation of `rick.jpg` file in your HOME directory!*

Authentication
--------------

- **Basic Auth**: Using the given middleware as demonstrated by the `BasicAuthHttpEndpoint`.
- **OAuth 2**: Using GitHub as demonstrated by the `GitHubHttpEndpoint`.

-----------------------------------------------------------------------------

fs2 examples
============

In the fs2 package you'll find some practical examples of the few things it's possible to build with this powerful streaming library. This might serve as a starting point, your creativity will do the rest.

fs2.async package
------------------

Apart from the use of the three core types `Stream[F, O]`, `Pipe[F, I, O]` and `Sink[F, I]` you'll find examples of use of the following types:

- `Topic[F, A]`
- `Signal[F, A]`
- `Queue[F, A]`
- `Ref[F, A]`
- `Promise[F, A]`
- `Semaphore[F]`

In addition to the use of some other functions useful in Parallel and Concurrent scenarios.



================================================
FILE: build.sbt
================================================
import Dependencies._

lazy val root = (project in file(".")).
  settings(
    inThisBuild(List(
      organization := "com.github.gvolpe",
      scalaVersion := "2.12.4",
      version      := "0.1.0-SNAPSHOT",
      scalacOptions := Seq(
        "-deprecation",
        "-encoding",
        "UTF-8",
        "-feature",
        "-language:existentials",
        "-language:higherKinds",
        "-Ypartial-unification"
      )    
    )),
    name := "Advanced Http4s",
    libraryDependencies ++= Seq(
      Libraries.catsEffect,
      Libraries.monix,
      Libraries.fs2Core,
      Libraries.http4sServer,
      Libraries.http4sClient,
      Libraries.http4sDsl,
      Libraries.http4sCirce,
      Libraries.http4sXml,
      Libraries.circeCore,
      Libraries.circeGeneric,
      Libraries.typesafeConfig,
      Libraries.logback,
      Libraries.scalaTest,
      Libraries.scalaCheck
    ),
    addCompilerPlugin("org.spire-math" % "kind-projector" % "0.9.5" cross CrossVersion.binary)
  )


================================================
FILE: project/Dependencies.scala
================================================
import sbt._

object Dependencies {

  object Versions {
    val CatsEffect  = "0.8"
    val Monix       = "3.0.0-M3"
    val Fs2         = "0.10.2"
    val Http4s      = "0.18.1"
    val Tsec        = "0.0.1-M9"
    val Circe       = "0.9.1"
    val ScalaTest   = "3.0.4"
    val ScalaCheck  = "1.13.4"
    val Logback     = "1.2.1"
    val TypesafeCfg = "1.3.1"
  }

  object Libraries {
    lazy val catsEffect     = "org.typelevel"       %% "cats-effect"                  % Versions.CatsEffect
    lazy val monix          = "io.monix"            %% "monix"                        % Versions.Monix

    lazy val fs2Core        = "co.fs2"              %% "fs2-core"                     % Versions.Fs2
    lazy val fs2IO          = "co.fs2"              %% "fs2-io"                       % Versions.Fs2

    lazy val http4sServer   = "org.http4s"          %% "http4s-blaze-server"          % Versions.Http4s
    lazy val http4sClient   = "org.http4s"          %% "http4s-blaze-client"          % Versions.Http4s
    lazy val http4sDsl      = "org.http4s"          %% "http4s-dsl"                   % Versions.Http4s
    lazy val http4sCirce    = "org.http4s"          %% "http4s-circe"                 % Versions.Http4s
    lazy val http4sXml      = "org.http4s"          %% "http4s-scala-xml"             % Versions.Http4s

    lazy val tsecJwtMac     = "io.github.jmcardon"  %% "tsec-jwt-mac"                 % Versions.Tsec

    lazy val circeCore      = "io.circe"            %% "circe-core"                   % Versions.Circe
    lazy val circeGeneric   = "io.circe"            %% "circe-generic"                % Versions.Circe

    lazy val typesafeConfig = "com.typesafe"        %  "config"                       % Versions.TypesafeCfg
    lazy val logback        = "ch.qos.logback"      %  "logback-classic"              % Versions.Logback

    lazy val scalaTest      = "org.scalatest"       %% "scalatest"                    % Versions.ScalaTest   % Test
    lazy val scalaCheck     = "org.scalacheck"      %% "scalacheck"                   % Versions.ScalaCheck  % Test
  }

}


================================================
FILE: project/build.properties
================================================
sbt.version=1.1.0


================================================
FILE: src/main/scala/com/github/gvolpe/fs2/Counter.scala
================================================
package com.github.gvolpe.fs2

import cats.effect.{Effect, IO}
import fs2.StreamApp.ExitCode
import fs2.async.Ref
import fs2.{Scheduler, Sink, Stream, StreamApp, async}

import scala.concurrent.ExecutionContext.Implicits.global

object CounterApp extends Counter[IO]

/**
  * Concurrent counter that demonstrates the use of [[fs2.async.Ref]].
  *
  * The workers will concurrently run and modify the value of the Ref so this is one
  * possible outcome showing "#worker >> currentCount":
  *
  * #1 >> 0
  * #3 >> 0
  * #2 >> 0
  * #1 >> 2
  * #2 >> 3
  * #3 >> 3
  * */
class Counter[F[_] : Effect] extends StreamApp[F] {

  override def stream(args: List[String], requestShutdown: F[Unit]): fs2.Stream[F, ExitCode] =
    Scheduler(corePoolSize = 10).flatMap { implicit S =>
      for {
        ref <- Stream.eval(async.refOf[F, Int](0))
        w1  = new Worker[F](1, ref)
        w2  = new Worker[F](2, ref)
        w3  = new Worker[F](3, ref)
        ec  <- Stream(w1.start, w2.start, w3.start).join(3).drain ++ Stream.emit(ExitCode.Success)
      } yield ec
    }

}

class Worker[F[_]](number: Int, ref: Ref[F, Int])
                  (implicit F: Effect[F]) {

  private val sink: Sink[F, Int] = _.evalMap(n => F.delay(println(s"#$number >> $n")))

  def start: Stream[F, Unit] =
    for {
      _ <- Stream.eval(ref.get) to sink
      _ <- Stream.eval(ref.modify(_ + 1))
      _ <- Stream.eval(ref.get) to sink
    } yield ()

}


================================================
FILE: src/main/scala/com/github/gvolpe/fs2/Fifo.scala
================================================
package com.github.gvolpe.fs2

import cats.effect.{Effect, IO}
import fs2.StreamApp.ExitCode
import fs2.async.mutable.Queue
import fs2.{Scheduler, Stream, StreamApp, async}

import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration._

object FifoApp extends Fifo[IO]

/**
  * Represents a FIFO (First IN First OUT) system built on top of two [[fs2.async.mutable.Queue]].
  *
  * q1 has a buffer size of 1 while q2 has a buffer size of 100 so you will notice the buffering when
  * pulling elements out of the q2.
  * */
class Fifo[F[_]: Effect] extends StreamApp[F] {

  override def stream(args: List[String], requestShutdown: F[Unit]): fs2.Stream[F, ExitCode] =
    Scheduler(corePoolSize = 4).flatMap { implicit S =>
      for {
        q1 <- Stream.eval(async.boundedQueue[F, Int](1))
        q2 <- Stream.eval(async.boundedQueue[F, Int](100))
        bp = new Buffering[F](q1, q2)
        ec <- S.delay(Stream.emit(ExitCode.Success).covary[F], 5.seconds) concurrently bp.start.drain
      } yield ec
    }

}

class Buffering[F[_]](q1: Queue[F, Int], q2: Queue[F, Int])(implicit F: Effect[F]) {

  def start: Stream[F, Unit] =
    Stream(
      Stream.range(0, 1000).covary[F] to q1.enqueue,
      q1.dequeue to q2.enqueue,
      //.map won't work here as you're trying to map a pure value with a side effect. Use `evalMap` instead.
      q2.dequeue.evalMap(n => F.delay(println(s"Pulling out $n from Queue #2")))
    ).join(3)

}

================================================
FILE: src/main/scala/com/github/gvolpe/fs2/Once.scala
================================================
package com.github.gvolpe.fs2

import cats.effect.{Effect, IO}
import fs2.StreamApp.ExitCode
import fs2.async.Promise
import fs2.{Scheduler, Stream, StreamApp, async}

import scala.concurrent.ExecutionContext.Implicits.global

object OnceApp extends Once[IO]

/**
  * Demonstrate the use of [[fs2.async.Promise]]
  *
  * Two processes will try to complete the promise at the same time but only one will succeed,
  * completing the promise exactly once.
  * The loser one will raise an error when trying to complete a promise already completed,
  * that's why we call `attempt` on the evaluation.
  *
  * Notice that the loser process will remain running in the background and the program will
  * end on completion of all of the inner streams.
  *
  * So it's a "race" in the sense that both processes will try to complete the promise at the
  * same time but conceptually is different from "race". So for example, if you schedule one
  * of the processes to run in 10 seconds from now, then the entire program will finish after
  * 10 seconds and you can know for sure that the process completing the promise is going to
  * be the first one.
  * */
class Once[F[_]: Effect] extends StreamApp[F] {

  override def stream(args: List[String], requestShutdown: F[Unit]): fs2.Stream[F, ExitCode] =
    Scheduler(corePoolSize = 4).flatMap { implicit scheduler =>
      for {
        p <- Stream.eval(async.promise[F, Int])
        e <- new ConcurrentCompletion[F](p).start
      } yield e
    }

}

class ConcurrentCompletion[F[_]](p: Promise[F, Int])(implicit F: Effect[F]) {

  private def attemptPromiseCompletion(n: Int): Stream[F, Unit] =
    Stream.eval(p.complete(n)).attempt.drain

  def start: Stream[F, ExitCode] =
    Stream(
      attemptPromiseCompletion(1),
      attemptPromiseCompletion(2),
      Stream.eval(p.get).evalMap(n => F.delay(println(s"Result: $n")))
    ).join(3).drain ++ Stream.emit(ExitCode.Success)

}


================================================
FILE: src/main/scala/com/github/gvolpe/fs2/PubSub.scala
================================================
package com.github.gvolpe.fs2

import cats.effect.{Effect, IO}
import fs2.StreamApp.ExitCode
import fs2.async.mutable.{Signal, Topic}
import fs2.{Scheduler, Sink, Stream, StreamApp, async}

import scala.concurrent.duration._
import scala.concurrent.ExecutionContext.Implicits.global

object PubSubApp extends PubSub[IO]

/**
  * Single Publisher / Multiple Subscribers application implemented on top of
  * [[fs2.async.mutable.Topic]] and [[fs2.async.mutable.Signal]].
  *
  * The program ends after 15 seconds when the signal interrupts the publishing of more events
  * given that the final streaming merge halts on the end of its left stream (the publisher).
  *
  * - Subscriber #1 should receive 15 events + the initial empty event
  * - Subscriber #2 should receive 10 events
  * - Subscriber #3 should receive 5 events
  * */
class PubSub[F[_]: Effect] extends StreamApp[F] {

  override def stream(args: List[String], requestShutdown: F[Unit]): fs2.Stream[F, ExitCode] =
    Scheduler(corePoolSize = 4).flatMap { implicit S =>
      for {
        topic     <- Stream.eval(async.topic[F, Event](Event("")))
        signal    <- Stream.eval(async.signalOf[F, Boolean](false))
        service   = new EventService[F](topic, signal)
        exitCode  <- Stream(
                      S.delay(Stream.eval(signal.set(true)), 15.seconds),
                      service.startPublisher concurrently service.startSubscribers
                    ).join(2).drain ++ Stream.emit(ExitCode.Success)
      } yield exitCode
    }

}

class EventService[F[_]](eventsTopic: Topic[F, Event],
                         interrupter: Signal[F, Boolean])
                        (implicit F: Effect[F], S: Scheduler) {

  // Publishing events every one second until signaling interruption
  def startPublisher: Stream[F, Unit] =
    S.awakeEvery(1.second).flatMap { _ =>
      val event = Event(System.currentTimeMillis().toString)
      Stream.eval(eventsTopic.publish1(event))
    }.interruptWhen(interrupter)

  // Creating 3 subscribers in a different period of time and join them to run concurrently
  def startSubscribers: Stream[F, Unit] = {
    val s1: Stream[F, Event] = eventsTopic.subscribe(10)
    val s2: Stream[F, Event] = S.delay(eventsTopic.subscribe(10), 5.seconds)
    val s3: Stream[F, Event] = S.delay(eventsTopic.subscribe(10), 10.seconds)

    def sink(subscriberNumber: Int): Sink[F, Event] =
      _.evalMap(e => F.delay(println(s"Subscriber #$subscriberNumber processing event: $e")))

    Stream(s1 to sink(1), s2 to sink(2), s3 to sink(3)).join(3)
  }

}

================================================
FILE: src/main/scala/com/github/gvolpe/fs2/Resources.scala
================================================
package com.github.gvolpe.fs2

import cats.effect.{Effect, IO}
import cats.syntax.functor._
import fs2.StreamApp.ExitCode
import fs2.async.mutable.Semaphore
import fs2.{Scheduler, Stream, StreamApp, async}

import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration._

object ResourcesApp extends Resources[IO]

/**
  * It demonstrates one of the possible uses of [[fs2.async.mutable.Semaphore]]
  *
  * Three processes are trying to access a shared resource at the same time but only one at
  * a time will be granted access and the next process have to wait until the resource gets
  * available again (availability is one as indicated by the semaphore counter).
  *
  * R1, R2 & R3 will request access of the precious resource concurrently so this could be
  * one possible outcome:
  *
  * R1 >> Availability: 1
  * R2 >> Availability: 1
  * R2 >> Started | Availability: 0
  * R3 >> Availability: 0
  * --------------------------------
  * R1 >> Started | Availability: 0
  * R2 >> Done | Availability: 0
  * --------------------------------
  * R3 >> Started | Availability: 0
  * R1 >> Done | Availability: 0
  * --------------------------------
  * R3 >> Done | Availability: 1
  *
  * This means when R1 and R2 requested the availability it was one and R2 was faster in
  * getting access to the resource so it started processing. R3 was the slowest and saw
  * that there was no availability from the beginning.
  *
  * Once R2 was done R1 started processing immediately showing no availability.
  *
  * Once R1 was done R3 started processing immediately showing no availability.
  *
  * Finally, R3 was done showing an availability of one once again.
  * */
class Resources[F[_]: Effect] extends StreamApp[F] {

  override def stream(args: List[String], requestShutdown: F[Unit]): fs2.Stream[F, ExitCode] =
    Scheduler(corePoolSize = 4).flatMap { implicit scheduler =>
      for {
        s   <- Stream.eval(async.semaphore[F](1))
        r1  = new PreciousResource[F]("R1", s)
        r2  = new PreciousResource[F]("R2", s)
        r3  = new PreciousResource[F]("R3", s)
        ec  <- Stream(r1.use, r2.use, r3.use).join(3).drain ++ Stream.emit(ExitCode.Success)
      } yield ec
    }

}

class PreciousResource[F[_]: Effect](name: String, s: Semaphore[F])
                                    (implicit S: Scheduler) {

  def use: Stream[F, Unit] =
    for {
      _ <- Stream.eval(s.available.map(a => println(s"$name >> Availability: $a")))
      _ <- Stream.eval(s.decrement)
      _ <- Stream.eval(s.available.map(a => println(s"$name >> Started | Availability: $a")))
      _ <- S.sleep(3.seconds)
      _ <- Stream.eval(s.increment)
      _ <- Stream.eval(s.available.map(a => println(s"$name >> Done | Availability: $a")))
    } yield ()

}

================================================
FILE: src/main/scala/com/github/gvolpe/fs2/package.scala
================================================
package com.github.gvolpe

package object fs2 {
  case class Event(value: String) extends AnyVal
}


================================================
FILE: src/main/scala/com/github/gvolpe/http4s/StreamUtils.scala
================================================
package com.github.gvolpe.http4s

import cats.effect.Sync
import fs2.Stream

trait StreamUtils[F[_]] {
  def evalF[A](thunk: => A)(implicit F: Sync[F]): Stream[F, A] = Stream.eval(F.delay(thunk))
  def putStrLn(value: String)(implicit F: Sync[F]): Stream[F, Unit] = evalF(println(value))
  def putStr(value: String)(implicit F: Sync[F]): Stream[F, Unit] = evalF(print(value))
  def env(name: String)(implicit F: Sync[F]): Stream[F, Option[String]] = evalF(sys.env.get(name))
  def error(msg: String): Stream[F, String] = Stream.raiseError[String](new Exception(msg)).covary[F]
}

object StreamUtils {
  implicit def syncInstance[F[_]: Sync]: StreamUtils[F] = new StreamUtils[F] {}
}

================================================
FILE: src/main/scala/com/github/gvolpe/http4s/client/MultipartClient.scala
================================================
package com.github.gvolpe.http4s.client

import java.net.URL

import cats.effect.Effect
import cats.syntax.flatMap._
import cats.syntax.functor._
import com.github.gvolpe.http4s.StreamUtils
import fs2.StreamApp.ExitCode
import fs2.{Scheduler, Stream, StreamApp}
import monix.eval.Task
import monix.execution.Scheduler.Implicits.global
import org.http4s.Method._
import org.http4s.client.blaze.Http1Client
import org.http4s.client.dsl.Http4sClientDsl
import org.http4s.headers.`Content-Type`
import org.http4s.multipart.{Multipart, Part}
import org.http4s.{MediaType, Uri}

object MultipartClient extends MultipartHttpClient[Task]

class MultipartHttpClient[F[_]](implicit F: Effect[F], S: StreamUtils[F]) extends StreamApp with Http4sClientDsl[F] {

  private val image: F[URL] = F.delay(getClass.getResource("/rick.jpg"))

  private def multipart(url: URL) = Multipart[F](
    Vector(
      Part.formData("name", "gvolpe"),
      Part.fileData("rick", url, `Content-Type`(MediaType.`image/png`))
    )
  )

  private val request =
    for {
      body <- image.map(multipart)
      req  <- POST(Uri.uri("http://localhost:8080/v1/multipart"), body)
    } yield req.replaceAllHeaders(body.headers)

  override def stream(args: List[String], requestShutdown: F[Unit]): Stream[F, ExitCode] = {
    Scheduler(corePoolSize = 2).flatMap { implicit scheduler =>
      for {
        client <- Http1Client.stream[F]()
        req    <- Stream.eval(request)
        value  <- Stream.eval(client.expect[String](req))
        _      <- S.evalF(println(value))
      } yield ()
    }.drain
  }

}


================================================
FILE: src/main/scala/com/github/gvolpe/http4s/client/StreamClient.scala
================================================
package com.github.gvolpe.http4s.client

import cats.effect.Effect
import com.github.gvolpe.http4s.StreamUtils
import fs2.StreamApp.ExitCode
import fs2.{Stream, StreamApp}
import io.circe.Json
import jawn.Facade
import monix.eval.Task
import monix.execution.Scheduler.Implicits.global
import org.http4s.client.blaze.Http1Client
import org.http4s.{Request, Uri}

object StreamClient extends HttpClient[Task]

class HttpClient[F[_]](implicit F: Effect[F], S: StreamUtils[F]) extends StreamApp {

  implicit val jsonFacade: Facade[Json] = io.circe.jawn.CirceSupportParser.facade

  override def stream(args: List[String], requestShutdown: F[Unit]): Stream[F, ExitCode] = {
    Http1Client.stream[F]().flatMap { client =>
      val request = Request[F](uri = Uri.uri("http://localhost:8080/v1/dirs?depth=3"))
      for {
        response <- client.streaming(request)(_.body.chunks.through(fs2.text.utf8DecodeC))
        _        <- S.putStr(response)
      } yield ()
    }.drain
  }

}


================================================
FILE: src/main/scala/com/github/gvolpe/http4s/server/Module.scala
================================================
package com.github.gvolpe.http4s.server

import cats.effect.Effect
import cats.syntax.semigroupk._ // For <+>
import com.github.gvolpe.http4s.server.endpoints._
import com.github.gvolpe.http4s.server.endpoints.auth.{BasicAuthHttpEndpoint, GitHubHttpEndpoint}
import com.github.gvolpe.http4s.server.service.{FileService, GitHubService}
import fs2.Scheduler
import org.http4s.HttpService
import org.http4s.client.Client
import org.http4s.server.HttpMiddleware
import org.http4s.server.middleware.{AutoSlash, ChunkAggregator, GZip, Timeout}

import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration._

class Module[F[_]](client: Client[F])(implicit F: Effect[F], S: Scheduler) {

  private val fileService = new FileService[F]

  private val gitHubService = new GitHubService[F](client)

  def middleware: HttpMiddleware[F] = {
    {(service: HttpService[F]) => GZip(service)(F)} compose
      { service => AutoSlash(service)(F) }
  }

  val fileHttpEndpoint: HttpService[F] =
    new FileHttpEndpoint[F](fileService).service

  val nonStreamFileHttpEndpoint = ChunkAggregator(fileHttpEndpoint)

  private val hexNameHttpEndpoint: HttpService[F] =
    new HexNameHttpEndpoint[F].service

  private val compressedEndpoints: HttpService[F] =
    middleware(hexNameHttpEndpoint)

  private val timeoutHttpEndpoint: HttpService[F] =
    new TimeoutHttpEndpoint[F].service

  private val timeoutEndpoints: HttpService[F] =
    Timeout(1.second)(timeoutHttpEndpoint)

  private val mediaHttpEndpoint: HttpService[F] =
    new JsonXmlHttpEndpoint[F].service

  private val multipartHttpEndpoint: HttpService[F] =
    new MultipartHttpEndpoint[F](fileService).service

  private val gitHubHttpEndpoint: HttpService[F] =
    new GitHubHttpEndpoint[F](gitHubService).service

  val basicAuthHttpEndpoint: HttpService[F] =
    new BasicAuthHttpEndpoint[F].service

  // NOTE: If you mix services wrapped in `AuthMiddleware[F, ?]` the entire namespace will be protected.
  // You'll get 401 (Unauthorized) instead of 404 (Not found). Mount it separately as done in Server.
  val httpServices: HttpService[F] = (
    compressedEndpoints <+> timeoutEndpoints
    <+> mediaHttpEndpoint <+> multipartHttpEndpoint
    <+> gitHubHttpEndpoint
  )

}


================================================
FILE: src/main/scala/com/github/gvolpe/http4s/server/Server.scala
================================================
package com.github.gvolpe.http4s.server

import cats.effect.Effect
import fs2.StreamApp.ExitCode
import fs2.{Scheduler, Stream, StreamApp}
import monix.eval.Task
import monix.execution.Scheduler.Implicits.global
import org.http4s.client.blaze.Http1Client
import org.http4s.server.blaze.BlazeBuilder

object Server extends HttpServer[Task]

class HttpServer[F[_]](implicit F: Effect[F]) extends StreamApp[F] {

  override def stream(args: List[String], requestShutdown: F[Unit]): Stream[F, ExitCode] =
    Scheduler(corePoolSize = 2).flatMap { implicit scheduler =>
      for {
        client   <- Http1Client.stream[F]()
        ctx      <- Stream(new Module[F](client))
        exitCode <- BlazeBuilder[F]
                      .bindHttp(8080, "0.0.0.0")
                      .mountService(ctx.fileHttpEndpoint, s"/${endpoints.ApiVersion}")
                      .mountService(ctx.nonStreamFileHttpEndpoint, s"/${endpoints.ApiVersion}/nonstream")
                      .mountService(ctx.httpServices)
                      .mountService(ctx.basicAuthHttpEndpoint, s"/${endpoints.ApiVersion}/protected")
                      .serve
      } yield exitCode
    }

}


================================================
FILE: src/main/scala/com/github/gvolpe/http4s/server/endpoints/FileHttpEndpoint.scala
================================================
package com.github.gvolpe.http4s.server.endpoints

import cats.Monad
import com.github.gvolpe.http4s.server.service.FileService
import org.http4s._
import org.http4s.dsl.Http4sDsl

class FileHttpEndpoint[F[_] : Monad](fileService: FileService[F]) extends Http4sDsl[F] {

  object DepthQueryParamMatcher extends OptionalQueryParamDecoderMatcher[Int]("depth")

  val service: HttpService[F] = HttpService {
    case GET -> Root / "dirs" :? DepthQueryParamMatcher(depth) =>
      Ok(fileService.homeDirectories(depth))
  }

}

================================================
FILE: src/main/scala/com/github/gvolpe/http4s/server/endpoints/HexNameHttpEndpoint.scala
================================================
package com.github.gvolpe.http4s.server.endpoints

import cats.Monad
import org.http4s._
import org.http4s.dsl.Http4sDsl

class HexNameHttpEndpoint[F[_]: Monad] extends Http4sDsl[F] {

  object NameQueryParamMatcher extends QueryParamDecoderMatcher[String]("name")

  val service: HttpService[F] = HttpService {
    case GET -> Root / ApiVersion / "hex" :? NameQueryParamMatcher(name) =>
      Ok(name.getBytes("UTF-8").map("%02x".format(_)).mkString)
  }

}


================================================
FILE: src/main/scala/com/github/gvolpe/http4s/server/endpoints/JsonXmlHttpEndpoint.scala
================================================
package com.github.gvolpe.http4s.server.endpoints

import cats.effect.Effect
import cats.syntax.flatMap._
import io.circe.generic.auto._
import org.http4s._
import org.http4s.circe._
import org.http4s.dsl.Http4sDsl

// Docs: http://http4s.org/v0.18/entity/
class JsonXmlHttpEndpoint[F[_]: Effect] extends Http4sDsl[F] {

  implicit def jsonXmlDecoder: EntityDecoder[F, Person] = jsonOf[F, Person] orElse personXmlDecoder[F]

  val service: HttpService[F] = HttpService {
    case GET -> Root / ApiVersion / "media" =>
      Ok("Send either json or xml via POST method. Eg: \n{\n  \"name\": \"gvolpe\",\n  \"age\": 30\n}\n or \n <person>\n  <name>gvolpe</name>\n  <age>30</age>\n</person>")

    case req @ POST -> Root / ApiVersion / "media" =>
      req.as[Person].flatMap { person =>
        Ok(s"Successfully decoded person: ${person.name}")
      }
  }

}


================================================
FILE: src/main/scala/com/github/gvolpe/http4s/server/endpoints/MultipartHttpEndpoint.scala
================================================
package com.github.gvolpe.http4s.server.endpoints

import cats.effect.Sync
import cats.implicits._
import com.github.gvolpe.http4s.server.service.FileService
import org.http4s._
import org.http4s.dsl.Http4sDsl
import org.http4s.multipart.Part

class MultipartHttpEndpoint[F[_]](fileService: FileService[F])
                                 (implicit F: Sync[F]) extends Http4sDsl[F] {

  val service: HttpService[F] = HttpService {
    case GET -> Root / ApiVersion / "multipart" =>
      Ok("Send a file (image, sound, etc) via POST Method")

    case req @ POST -> Root / ApiVersion / "multipart" =>
      req.decodeWith(multipart[F], strict = true) { response =>
        def filterFileTypes(part: Part[F]): Boolean = {
          part.headers.exists(_.value.contains("filename"))
        }

        val stream = response.parts.filter(filterFileTypes).traverse(fileService.store)

        Ok(stream.map(_ => s"Multipart file parsed successfully > ${response.parts}"))
      }
  }

}


================================================
FILE: src/main/scala/com/github/gvolpe/http4s/server/endpoints/TimeoutHttpEndpoint.scala
================================================
package com.github.gvolpe.http4s.server.endpoints

import java.util.concurrent.TimeUnit

import cats.effect.Async
import fs2.Scheduler
import org.http4s._
import org.http4s.dsl.Http4sDsl

import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration.FiniteDuration
import scala.util.Random

class TimeoutHttpEndpoint[F[_]](implicit F: Async[F], S: Scheduler) extends Http4sDsl[F] {

  val service: HttpService[F] = HttpService {
    case GET -> Root / ApiVersion / "timeout" =>
      val randomDuration = FiniteDuration(Random.nextInt(3) * 1000L, TimeUnit.MILLISECONDS)
      S.effect.delay(Ok("delayed response"), randomDuration)
  }

}


================================================
FILE: src/main/scala/com/github/gvolpe/http4s/server/endpoints/auth/AuthRepository.scala
================================================
package com.github.gvolpe.http4s.server.endpoints.auth

import cats.effect.Sync
import org.http4s.BasicCredentials

trait AuthRepository[F[_], A] {
  def persist(entity: A): F[Unit]
  def find(entity: A): F[Option[A]]
}

object AuthRepository {

  implicit def authUserRepo[F[_]](implicit F: Sync[F]): AuthRepository[F, BasicCredentials] =
    new AuthRepository[F, BasicCredentials] {
      private val storage = scala.collection.mutable.Set[BasicCredentials](
        BasicCredentials("gvolpe", "123456")
      )
      override def persist(entity: BasicCredentials): F[Unit] = F.delay(storage.add(entity))
      override def find(entity: BasicCredentials): F[Option[BasicCredentials]] = F.delay(storage.find(_ == entity))
    }

}


================================================
FILE: src/main/scala/com/github/gvolpe/http4s/server/endpoints/auth/BasicAuthHttpEndpoint.scala
================================================
package com.github.gvolpe.http4s.server.endpoints.auth

import cats.effect.Sync
import com.github.gvolpe.http4s.server.endpoints.ApiVersion
import org.http4s._
import org.http4s.dsl.Http4sDsl
import org.http4s.server.AuthMiddleware
import org.http4s.server.middleware.authentication.BasicAuth

// Use this header --> Authorization: Basic Z3ZvbHBlOjEyMzQ1Ng==
class BasicAuthHttpEndpoint[F[_]](implicit F: Sync[F], R: AuthRepository[F, BasicCredentials]) extends Http4sDsl[F] {

  private val authedService: AuthedService[BasicCredentials, F] = AuthedService {
    case GET -> Root as user =>
      Ok(s"Access Granted: ${user.username}")
  }

  private val authMiddleware: AuthMiddleware[F, BasicCredentials] =
    BasicAuth[F, BasicCredentials]("Protected Realm", R.find)

  val service: HttpService[F] = authMiddleware(authedService)

}


================================================
FILE: src/main/scala/com/github/gvolpe/http4s/server/endpoints/auth/GitHubHttpEndpoint.scala
================================================
package com.github.gvolpe.http4s.server.endpoints.auth

import cats.effect.Sync
import cats.syntax.flatMap._
import cats.syntax.functor._
import com.github.gvolpe.http4s.server.endpoints.ApiVersion
import com.github.gvolpe.http4s.server.service.GitHubService
import org.http4s._
import org.http4s.dsl.Http4sDsl

class GitHubHttpEndpoint[F[_]](gitHubService: GitHubService[F])
                              (implicit F: Sync[F]) extends Http4sDsl[F] {

  object CodeQuery extends QueryParamDecoderMatcher[String]("code")
  object StateQuery extends QueryParamDecoderMatcher[String]("state")

  val service: HttpService[F] = HttpService {
    case GET -> Root / ApiVersion / "github" =>
      Ok(gitHubService.authorize)

    // OAuth2 Callback URI
    case GET -> Root / ApiVersion / "login" / "github" :? CodeQuery(code) :? StateQuery(state) =>
      Ok(gitHubService.accessToken(code, state).flatMap(gitHubService.userData))
        .map(_.putHeaders(Header("Content-Type", "application/json")))
  }

}


================================================
FILE: src/main/scala/com/github/gvolpe/http4s/server/endpoints/package.scala
================================================
package com.github.gvolpe.http4s.server

import cats.effect.Sync
import org.http4s.EntityDecoder

import scala.xml._

package object endpoints {
  val ApiVersion = "v1"

  case class Person(name: String, age: Int)

  /**
    * XML Example for Person:
    *
    * <person>
    *   <name>gvolpe</name>
    *   <age>30</age>
    * </person>
    * */
  object Person {
    def fromXml(elem: Elem): Person = {
      val name = (elem \\ "name").text
      val age  = (elem \\ "age").text
      Person(name, age.toInt)
    }
  }

  def personXmlDecoder[F[_]: Sync]: EntityDecoder[F, Person] =
    org.http4s.scalaxml.xml[F].map(Person.fromXml)

}


================================================
FILE: src/main/scala/com/github/gvolpe/http4s/server/service/FileService.scala
================================================
package com.github.gvolpe.http4s.server.service

import java.io.File
import java.nio.file.Paths

import cats.effect.Effect
import com.github.gvolpe.http4s.StreamUtils
import fs2.Stream
import org.http4s.multipart.Part

class FileService[F[_]](implicit F: Effect[F], S: StreamUtils[F]) {

  def homeDirectories(depth: Option[Int]): Stream[F, String] =
    S.env("HOME").flatMap { maybePath =>
      val ifEmpty = S.error("HOME environment variable not found!")
      maybePath.fold(ifEmpty)(directories(_, depth.getOrElse(1)))
    }

  def directories(path: String, depth: Int): Stream[F, String] = {

    def dir(f: File, d: Int): Stream[F, File] = {
      val dirs = Stream.emits(f.listFiles().toSeq).filter(_.isDirectory).covary[F]

      if (d <= 0) Stream.empty
      else if (d == 1) dirs
      else dirs ++ dirs.flatMap(x => dir(x, d - 1))
    }

    S.evalF(new File(path)).flatMap { file =>
      dir(file, depth)
        .map(_.getName)
        .filter(!_.startsWith("."))
        .intersperse("\n")
    }
  }

  def store(part: Part[F]): Stream[F, Unit] =
    for {
      home      <- S.evalF(sys.env.getOrElse("HOME", "/tmp"))
      filename  <- S.evalF(part.filename.getOrElse("sample"))
      path      <- S.evalF(Paths.get(s"$home/$filename"))
      _         <- part.body to fs2.io.file.writeAll(path)
    } yield ()

}


================================================
FILE: src/main/scala/com/github/gvolpe/http4s/server/service/GitHubService.scala
================================================
package com.github.gvolpe.http4s.server.service

import cats.effect.Sync
import cats.syntax.functor._
import com.github.gvolpe.http4s.server.endpoints.ApiVersion
import fs2.Stream
import io.circe.generic.auto._
import org.http4s.circe._
import org.http4s.client.Client
import org.http4s.client.dsl.Http4sClientDsl
import org.http4s.{Header, Request, Uri}

// See: https://developer.github.com/apps/building-oauth-apps/authorization-options-for-oauth-apps/#web-application-flow
class GitHubService[F[_]: Sync](client: Client[F]) extends Http4sClientDsl[F] {

  // NEVER make this data public! This is just a demo!
  private val ClientId     = "959ea01cd3065cad274a"
  private val ClientSecret = "53901db46451977e6331432faa2616ba24bc2550"

  private val RedirectUri  = s"http://localhost:8080/$ApiVersion/login/github"

  case class AccessTokenResponse(access_token: String)

  val authorize: Stream[F, Byte] = {
    val uri = Uri.uri("https://github.com")
      .withPath("/login/oauth/authorize")
      .withQueryParam("client_id", ClientId)
      .withQueryParam("redirect_uri", RedirectUri)
      .withQueryParam("scopes", "public_repo")
      .withQueryParam("state", "test_api")

    client.streaming[Byte](Request[F](uri = uri))(_.body)
  }

  def accessToken(code: String, state: String): F[String] = {
    val uri = Uri.uri("https://github.com")
      .withPath("/login/oauth/access_token")
      .withQueryParam("client_id", ClientId)
      .withQueryParam("client_secret", ClientSecret)
      .withQueryParam("code", code)
      .withQueryParam("redirect_uri", RedirectUri)
      .withQueryParam("state", state)

    client.expect[AccessTokenResponse](Request[F](uri = uri))(jsonOf[F, AccessTokenResponse])
      .map(_.access_token)
  }

  def userData(accessToken: String): F[String] = {
    val request = Request[F](uri = Uri.uri("https://api.github.com/user"))
      .putHeaders(Header("Authorization", s"token $accessToken"))

    client.expect[String](request)
  }

}


================================================
FILE: src/test/scala/com/github/gvolpe/http4s/IOAssertion.scala
================================================
package com.github.gvolpe.http4s

import cats.effect.IO

object IOAssertion {
  def apply[A](ioa: IO[A]): A = ioa.unsafeRunSync()
}


================================================
FILE: src/test/scala/com/github/gvolpe/http4s/server/endpoints/HexNameHttpEndpointSpec.scala
================================================
package com.github.gvolpe.http4s.server.endpoints

import cats.effect.IO
import com.github.gvolpe.http4s.IOAssertion
import org.http4s.server.middleware.GZip
import org.http4s.{Header, HttpService, Query, Request, Uri}
import org.scalatest.FunSuite

// Docs: http://http4s.org/v0.18/gzip/
class HexNameHttpEndpointSpec extends FunSuite {

  private val httpService: HttpService[IO] = GZip(new HexNameHttpEndpoint[IO].service)

  private val CompressedLength  = 74
  private val NormalLength      = 88

  private val request = Request[IO](uri =
    Uri(
      path = s"/$ApiVersion/hex",
      query = Query("name" -> Some("Scala is a really cool programming language!"))
    )
  )

  test("Compressed Response") {
    IOAssertion {
      val gzipHeader  = Header("Accept-Encoding", "gzip")
      val gzipRequest = request.putHeaders(gzipHeader)

      httpService(gzipRequest).value.flatMap { maybe =>
        maybe.fold(IO[Unit](fail("Empty response"))) { response =>
          response.as[String].map(r => assert(r.length == CompressedLength))
        }
      }
    }
  }

  test("Uncompressed Response") {
    IOAssertion {
      httpService(request).value.flatMap { maybe =>
        maybe.fold(IO[Unit](fail("Empty response"))) { response =>
          response.as[String].map(r => assert(r.length == NormalLength))
        }
      }
    }
  }

}


================================================
FILE: src/test/scala/com/github/gvolpe/http4s/server/endpoints/JsonXmlHttpEndpointSpec.scala
================================================
package com.github.gvolpe.http4s.server.endpoints

import cats.effect.IO
import com.github.gvolpe.http4s.IOAssertion
import org.http4s.{Header, HttpService, Method, Request, Status, Uri}
import org.scalatest.FunSuite

class JsonXmlHttpEndpointSpec extends FunSuite {

  private val httpService: HttpService[IO] = new JsonXmlHttpEndpoint[IO].service

  private val jsonPerson =
    """
      |{
      |  "name": "gvolpe",
      |  "age": 30
      |}
    """.stripMargin

  private val xmlPerson =
    """
      |<person>
      |  <name>gvolpe</name>
      |  <age>30</age>
      |</person>
    """.stripMargin

  private val request = Request[IO](method = Method.POST, uri = Uri(path = s"/$ApiVersion/media"))

  test("json is decoded") {
    IOAssertion {
      val bodyRequest = request.withBody[String](jsonPerson)

      bodyRequest.flatMap { req =>
        httpService(req.putHeaders(Header("Content-Type", "application/json"))).value.map { maybe =>
          maybe.fold(fail("Empty response")) { response =>
            assert(response.status == Status.Ok)
          }
        }
      }
    }
  }

  test("xml is decoded") {
    IOAssertion {
      val bodyRequest = request.withBody[String](xmlPerson)

      bodyRequest.flatMap { req =>
        httpService(req.putHeaders(Header("Content-Type", "application/xml"))).value.map { maybe =>
          maybe.fold(fail("Empty response")) { response =>
            assert(response.status == Status.Ok)
          }
        }
      }
    }
  }

  test("decoding fails, no Content Type") {
    IOAssertion {
      val bodyRequest = request.withBody[String](jsonPerson)

      bodyRequest.flatMap { req =>
        httpService(req).value.attempt.map {
          case Left(e)  => assert(e.getMessage == "Malformed message body: Invalid XML")
          case Right(_) => fail("Got a response when a failure was expected")
        }
      }

      // Using `req.decode` gives you a response, using `req.as` throws an exception
      // https://gitter.im/http4s/http4s?at=5a964662758c233504cc0fec
//      bodyRequest.flatMap { req =>
//        httpService(req).value.map { maybe =>
//          maybe.fold(fail("Empty response")) { response =>
//            assert(response.status == Status.BadRequest)
//          }
//        }
//      }
    }
  }

}